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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be 8vaBablftund6rthopravistanaof$7CFR1.138(8X In no event however* may a reply be timely Red 
after SIX (B) MONTHS from the maStrtg date of this communication. 

• If the period for repty specified above tetesst^ wiH be considered Umefy. 

- If NO period for repty is specified above, the maximum statutory period wBl appfy and wOl exptro SIX (6) MONTHS from the maSng date of ffris communfcatfort 

- Fafiure to reply wahin the set or extended period for rejtywffl. by statute, cause (heap (35US.C. § 133). 
Any reply received by the Office later than trmmonto after ^rnaiSnQ date c4 this comrmmicatton. even if Bmery fifed, may reduce any 
earned patent term adjustment See 37 CFR 1.704(b). 

Status 

1)B] Responsive to communication's) filed on 10 September 2001 . 
2a)D This action is FINAL. 2b)H This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayte, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) E3 Claim(s) 1-26 and 28-31 is/are pending in the application. 

4a) Of the above claim(s) 27 and 32 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claimfe) 1-5. 7-13. 15. 18-20.24-26 and 28-31 is/are rejected. 

7) El Clatm(s) 6.14.16.17.21-23 is/are objected to. 

B)Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9)13 The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 10 September 2001 is/are: a)D accepted or b)l3 objected to by the Examiner. 

Applicant may not request thai any objection to the drawings) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )£3 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (0- 
a® All b)Q Some *c)D None of: 

1 .§3 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Claims 1 - 26 and 28-31 have been examined. 
In a Preliminary Amendment 

Claims 1-26 and 28-3 1 were amended 

Claims 27 and 32 were cancelled 

Priority 

1 . Receipt is acknowledged of papers submitted under 35 U.S.C 1 1 9(a)-(d), which papers 
have been placed of record in the file. 

Information Disclosure Statement 

2. The Information Disclosure Statement (IDS) filed December 6, 2001 has been 
considered. The reference in French could not be considered. 

Oath/Declaration 

3. Applicant has elected to use an outdated version of 37 CFR 1.56 u (b$ amended effective 
March 16, 1992)". Applicant should use the current form on the USPTO.GOV website when 

. submitting a new Declaration. 

Drawings 

4. New corrected drawings in compliance with 37 CFR 1 . 121(d) are required in this 
application because they are fuzzy and hard to read and will not display properly in a U.S. 
Patent Applicant is advised to employ the services of a competent patent draftsperson outside 
the Office, as the U.S. Patent and Trademark Office no longer prepares new drawings. The 
corrected drawings are required in reply to the Office action to avoid abandonment of the 
application. The requirement for corrected drawings will not be held in abeyance. 
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5. Figure 1 should be designated by a legend such as -Prior Art- because only that which 
is old is illustrated. See MPEP § 608.02(g). Corrected drawings in compliance with 37 CFR 
1.121(d) are required in reply to the Office action to avoid abandonment of the application. The 
replacement sheets) should be labeled "Replacement Sheet" in the page header (as per 37 CFR 
1 .84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted 
by the examiner, the applicant will be notified and informed of any required corrective action in 
the next Office action. The objection to the drawings will not be held in abeyance. 

Specification 

6. The abstract of the disclosure is objected to because must be on a separate page. 
Correction is required. See MPEP § 608.01(b). 

7. Preliminary amendment of September 10, 2001 has been entered. 

8. The disclosure is objected to because of the following informalities: The spelling of 
several words is not in the format for United States English, the European spelling of the 
following must be changed. 

European Spelling United States English Spelling 

"analysing" analyzing 
"analysed" analyzed 
"reinitialised" reinitialized 
Reinitialisation" reinitialization 
Correction will benefit the searching of U.S. Patent literature. 
Appropriate correction is required. 
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9. Page 5 of the Specification contains the acronym *TIBs" without the term being fully 
spelt out. On common meaning is 4i Secured hash standard, Federal Information Processing 
Standards Publication (FIBS) 180-1, May 1994". Clarification required with a change to the 
Specification. 

10. The use of the trademark "JAVA" has been noted in this application. It should be 
capitalized wherever it appears and be accompanied by the generic terminology. 

Although the use of trademarks is permissible in patent applications, the proprietary 
nature of the marks should be respected and every effort made to prevent their use in any manner 
which might adversely affect their validity as trademarks. 

1 1 . The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed 

Claim Rejections - 15 USC § 112 

12. The following is a quotation of the second paragraph of 35 U.S.C 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

13. Claims 8 - 10 are rejected under 35 U.S,C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. The problem is the Applicant states the program to be monitored (DATA) . The 
focus of the claim language should the functionality of the monitor program and how it handles 
the varies condition presented by the input as it is processed. The Specification clearly supports 
what the Applicant is attempting to claim. This claim as written is indefinite. Dependent claims 
are also rejected merely because they are dependent on claim 8. 
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Claim 8 

A method according to Claim 1 wherein, when the program to be monitored provides for at least 
one jump, the monitoring method is applied separately to sets of instructions in the program 
which do not include jumps between two successive instructions. 

Claim Rejections - 35 USC§ 102 

14. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless * 

(b) die invention was patented or.described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in die United States. 

15. Claims 1 -5, 7-13, 15, 18-20, 24-26, 28, 30 and 31 are rejected under 35 U.S.C. 102(b) as 
being anticipated by USPN # 4,266,272 Berglund et al (IDS). 

The environment of the invention JAVACARD is not claimed but is vastly different than the 
environment of the IDS reference 
Claim Interpretation 

The control circuitry in the reference IDS is performing the monitor function of the claimed 
invention. 

Claim 1 

IDS anticipates a method for monitoring progress with the execution of a linear sequence of 
instructions in a computer program (IDS, Abstract, control circuitry ), comprising the steps of 
analysing the sequence of instructions transmitted to a processor intended to execute the program 
being monitored by extracting a data item from each instruction transmitted to the processor 
(IDS, Abstract, check word ) and performing a calculation on said data item (IDS, Abstract, 
dynamically calculated ), and verifying, the result of this analysis by comparing the result of said 
calculation to reference data (IDS, Abstract, local storage register vs. ALU ), recorded with said 
program, wherein the reference data comprises a value pre-established so as to correspond to the 
result of the analysis produced during the monitoring method only if all the instructions in the 
sequence of instructions have actually been analysed during the running of the program (IDS, 
Abstract, control storage ). 
Claim Interpretation 

The limitation "of a linear sequence of instructions" is not given patentable weight because it is 
dependent on the form of the input. Not part of the invention. It is treated as data. 
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Claim 2 

A method according to Claim 1 , wherein the verification of the result of the analysis is caused by 
an instruction placed at a predetermined location in the program to be monitored ( as per claim 1 
a register is a predetermined location), said instruction containing the reference data relating to a 
set of instructions whose correct execution is to be monitored (registers are inherently related to 
the instruction being processed). 

Claim 3 

A method according to Claim 1 wherein, when the instructions of the set of instructions to be 
monitored are in the form of a value, said analysis of the instructions is carried out by using these 
instructions as a numerical value. (Interpretation - all values are in binary format - this is 
inherent). 

Claim 4 

A method according to Claim 1, comprising the steps of: 

- during the preparation of the program to be monitored ( as per claim 1): 

- incorporating, in at least one predetermined location in a sequence of instructions (as per claim 
1) in the program, a reference value established according to a predetermined rule applied to 
identifiable data in each instruction to be monitored ( as per claim 1, identification of words), 
and during the execution of the program to be monitored ( as per claim 1): 

- obtaining said identifiable data in each instruction received for execution (IDS, fetch col 9, 10- 
30), 

- applying said predetermined rule to said identifiable data thus obtained in order to establish a 
verification value ( as per claim 1), and 

- verifying that this verification value actually corresponds to the reference value recorded with 
the program ( as per claim 1). 

Claims 

A method according to Claim 1 , further comprising a step of interrupting the flow of the program 
if the analysis reveals that the program being monitored has not been run as expected. (IDS, 
Figure #4, Result ERROR from result branch). . 

Claim 7 

A method according to Claim 1 wherein the set of instructions to be monitored does not include 
jumps in its expected flow. 
Claim Interpretation 

The limitation "set of instructions to be monitored does not include jumps in its expected flow" 
is not given patentable weight because it is dependent on the form of the input. Not part of the 
invention. It is treated as data. 

Claim 8 

A method according to Claim 1 wherein, when the program to be monitored provides for at least 
one jump, the monitoring method is applied separately to sets of instructions in the program 
which do not include jumps between two successive instructions ( IDS, col 9, lines 10 - 40). 
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Claim Interpretation 

The limitation "program to be monitored provides for at least one jump" is not given patentable 
weight because it is dependent on the form of the input Not part of the invention. It is treated as 
data. 

Claim 9 

A method according to Claim 8, wherein, when the program to be monitored includes an 
instruction for a jump dependent on the manipulated data, the monitoring method is implemented 
separately for a set of instructions which precedes the jump, and for at least one set of 
instructions which follows said jump. As per claim 8. 

Claim 10 

A method according to Claim 9, wherein, for a set of instructions providing for a jump, an 
instruction which controls this jump is integrated in said set of instructions for the purpose of 
obtaining a verification value for thus set of instructions before executing the jump instruction, 
as per claim 8. 

Claim 11 

A method according to Claim 1 wherein the analysis is reinitialised before each new monitoring 
of a sequence of instructions to be monitored. (IDS, cycle and incremented col 3, lines 40 - 60) 

Claim 12 

A method according to Claim 11, wherein the reinitialisation of the analysis of each new 
monitoring includes the step of erasing or replacing a verification value obtained during a 
previous analysis. As per claim 1 1 depending on cycle determination. 

Claim 13 

A method according to Claim 1 1 wherein the reinitialisation of the monitoring analysis is 
controlled by the software itself. (Interpretation - the control circuitry and software being 
executed has a functional relationship - This is deemed inherent and related to Examiner's note 
above) 

Claim 15 

A method according to Claim I wherein the analysis includes the step of calculating, for each 
instruction under consideration following a previous instruction, the result of an operation on 
both a value obtained of the instruction in question and the result obtained by the same operation 
performed on the previous instruction. As per claim 1. 

Claim 18 

A method according to Claim 1 wherein the analysis includes the step of obtaining a comparison 
value by calculating successive intermediate values as the data of the respective instructions are 
obtained, (IDS, Abstract, last sentence words is plural). 



Claim 19 
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A method according to Claim 1 wherein the analysis comprises a step of saving each data item 
necessary for verification, obtained from instructions in die set of instructions to be monitored as 
they are executed, and performing a calculation of a verification value from these data only at the 
necessary time, once all the necessary data have been obtained. ( as per claim 1 and details of 
fetch col 3, lines 10-30). 

Claim 20 

IDS anticipates a device for monitoring progress with the execution of a series of instructions of 
a computer program, comprising means for analysing the sequence of instructions transmitted to 
the processor intended to execute the program being monitored by extracting a data item from 
each instruction transmitted to the processor and performing a calculation on said data item, and 
means for verifying the result of this analysis by comparing the result of said calculation to 
reference data recorded with said program, wherein the reference data comprises a value pre- 
established so as to correspond to the result of the analysis produced during monitoring only if 
all the instructions in the sequence of instructions have actually been analysed during the running 
of the program. As per claim 1. 
Claim Interpretation 

The limitation "a series of instructions of a computer program" is not given patentable weight 
because it is dependent on the form of the input. Not part of the invention. It is treated as data. 

Claim 24 

A device according to Claim 20 that is integrated into a programmed device containing said 
program to be monitored. (IDS, Abstract, Control Circuitry). 

Claim 25 

A device according to Claim 20 that is integrated into a program execution device. (IDS, 
Abstract, Control Circuitry). 

Claim 26 

IDS anticipates a program execution device that executes a series of instructions of a computer 
program, comprising means for analysing the sequence of instructions transmitted for execution 
by extracting a data item from each instruction and performing a calculation on said data item, 
and means for verifying the result of this analysis by comparing the result of said calculation to 
reference data recorded with the program to be monitored, wherein the reference data comprises 
a value pre-established so as to correspond to the result of the analysis produced during 
monitoring only if all the instructions in the sequence of instructions have actually been analysed 
during the running of the program. As per claim 1 * 
Claim Interpretation 

A. The limitation "a series of instructions of a computer program" is not given patentable weight 
because it is dependent on the form of the input Not part of die invention. It is treated as data. 

B. In a similar fashion. The limitation "correspond to the result of the analysis produced during 
monitoring only if all the instructions in the sequence of instructions have actually been analysed 
dining the running of the program* 9 can be dependent on the input If die program is only s few 
statement which all statements are to execute the claim limitations are input dependent. The 
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claim limitations outside the fact a monitor function is present have not performed a non required 
step to distinguish it from a monitor. 

Claim 28 

IDS anticipates a programmed device containing a series of recorded instructions and a fixed 
memory containing reference data pre-established as a function of data contained in said 
instructions for analysis and verification of the sequence of instructions, wherein the reference 
data comprises a value pre-established so as to correspond to the result of the analysis produced 
during monitoring only if all the instructions in the sequence of instructions have actually been 
analyzed dining the running of the program, as per claim L 
Claim Interpretation 

A. The limitation "a series of recorded instructions" is not given patentable weight because it is 
dependent on the form of the input. Not part of the invention. It is treated as data. 

Claim 30 

A device according to Claim 28 wherein the reference data are recorded in the form of a 
prewired value or values fixed in memory . (IDS , Abstract, last sentence). 
Claim Interpretation 

The presence of the OR in the limitations, (he Examiner elects to reject the underlined limitation 
above* 

Claim 31 

A device for programming a programmed device according to Claim 28, comprising means for 
entering^ in at least one predetermined location in a sequence of instructions in the program, a 
reference value calculated according to a preestablished mode from data included in each 
instruction in a set of instructions whose execution is to be monitored. As per claim 1 . 

CUum Rejections - 35 USC§ 103 

16. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title* if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner tn which the invention was made. 

1 7. Claim 29 is rejected under 35 US.C. 103(a) as being unpatentable over IDS in view of 
USPN# 6,402,028. 



Claim 29 
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IDS teaches a device but IDS does not teach the device is a smart card, according to Claim 28, 
wherein said device is a smart card. USPN 6,402,028 teaches the production of Smart Cards 
where the logic is on the card, therefore, it would have been obvious to one of ordinary skill in 
the art at the time of invention, to combine IDS with 6,402,028 because logic control for Smart 
cards makes Smart Cards more reliable. 



Allowable Subject Matter 
18. Claims 6, 14, 16, 17 and 21 - 23 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. The bold and underlined limitations below indicate 
limitations not found in the prior art of record. 

Claim 6 

A method according to Claim 1, further comprising an invalidation step for future use of the 
device comprising the monitored program if said analysis reveals a predetermined number of 
times that the program being monitored has not run in the expected manner . 

Claim 14 

A method according to Claim 1 wherein the analysis produces a verification value obtained as 
the last value in a series of values which is made to change successively with the analysis of 
each of the analysed instructions of the set of instructions, thus making it possible to 
contain an internal state of the running of the monitoring method and to follow its changes* 

Claim 16 

A method according to Claim 1 wherein the analysis includes the step of recursively applying a 
hash function to values obtained of each monitored instruction, starting from a last 
initialisation performed. 

Claim 17 

A method according to Claim 1 wherein the analysis includes the step of making a verification 
value change by performing a redundancy calculation on aD the operating codes and the 
addresses executed since the last initialisation was carried out . 

Claim 21 

A device according to Claim 20, further including a register for recording intermediate results in 
a calculatio n in a chain carried out bv the analysis means in order to obtain a verification 
value. 
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Claim 22 

A device according to Claim 21, further comprising means for recording a predetermined value 
or resetting the register under the control of an instruction transmitted during the execution of a 
program to be monitored (Dependent on claim 21) 

Claim 23 

A device according to Claim 20, further comprising means for counting the number of 
unexpected events in die program being monitored, as determined by the analysis means, and 
means for invalidating the future use of the program to be monitored if this number 
reaches a predetermined threshold. 



Conclusion 

19, Hie prior art made of record and not relied upon is considered pertinent to applicant's 

disclosure. 

US Patent Literature 

A. 6,402,028 - Deals with mass production of Smart Cards Column 4 covers JAVACARD 
technology. 

B. 6,668,325 - Employs an obfuscation technique on a section of code. Environment is 
distributed. 

C. 5,974,549 - Monitor is implemented via Dynamic Link Library (DLL). 

D. 6,546,546 - Appears to be dependent on the extensible operating system disclosed 
(PARAMECIUM). 

E. 6,092,120 - Focus on class loaders. 

F. 6,327,700 - Based on Profile data. 

G. 6,557,168 - Monitor is included at class level not at low level as per disclosed invention. 
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H. 6,275,938 - The monitor environment runs at operating system level not processor level as 
disclosed invention. 



20. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Todd Ingberg whose telephone number is (571) 272-3723. Hie 
examiner can normally be reached on during the work week.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kakali Chaki can be reached on (571)272-3719. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). / 



Correspondence 




Todd Ingbetg / 
Primary Examiner 
Art Unit 2124 
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Building the First 
Java Examples 



Welcome.to Java 21 An ambitious agenda lie? before 
you: You're going to get a firm grip on Java program- 
ming, creating both powerful Java programs and Web 
pages, and you will take a guided tour through Java 2, There is < 
no more exciting programming package available. As you are ■ - V 
probably aware, the popularity of Java has skyrocketed as more 
and more people have seen how versatile and powerful it is, Web 
programmers have found it an excellent topi because it allows 
them to write programs that wfflnm on 
.^mput^ Th^havestarted using it to*i^teWd>f^^^ 
^•"'"'actualfyctoaomethmg. 1 * y - '■" --- H T ~- ; — ^vsr- - - 1 




4 Chapter One 



With Java, you will be able to display animation and images, accept 
mouse elides and text, use controls lite scrollbars and check boxes, print 
graphics, support pop-up menus, and even support additional windows 
and menu bars. 

Well start working on your Java skills right away-you wont need 
to wade through chapters of abstractions first We will concentrate on 
examples, on seeing things.from the programmer's point of view-on 
seeing Java at work. 

Java programs come two ways; as stand-alone applications and as 
small programs you can embed in Web pages, called applets. Of the two, 
applets are the most popular, and well concentrate primarily on them. 



Building the Hello Example 

The first example will be a simple one because right now we just want to 
get you started in Java without too many extra details to weigh you down. 
Vyou^tteate>^ v 
embed in a Web page, that will display the words "HeUoftom Java** 




, What's an Applet?. 

, .>:.: youctoemWinaWebp^su&t^ :~^-*k 
certain part of the Web page. On that part of the page, the applet can 

.... -'■^Arf^^.o^ 



Eachappletis given the amoimtof spaa (usually measured in ^ 
that it requests in atyeb page, such as the amount of space shewrn inFig- 
: ure LL (Soon ITl.show you hcraran ^>plet 'requests" space.) Ms is the 7 
space that the applet will use for its display. Well place the words 'Hello 
from Java!" in the applet, as shown in Figure 1.2. ; 



FIGURE 1 .1 : An applet requests space in a Web page. 
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Hello from Java! 




■ sji--' x.; .ftp 

d in pixels) ; 

Fhisis lbe m l. . |$ 
>nis "Hello r - W 



FIGURE 1 A Hello from Javal 

Creating the Hello Example 

*• ; -toseehoWtodotbis^ 
^ H _ . vuua me traditional first program m most Java 

. • P*Hc.v«1rf P»1nt(Graphict.'. 9 ) , ' . ! 

} - 9.dra«Strtng('He11o frt»i JavalCeo, 30 

to cHsk as.hellooava. «VPtd^n the text, save it 




NOTE 



hello, java, not Hello, java 



In general the name of the file will match exactly (including case) the 
name given in the "class" statement in the file; in this case, that is 
hello: 

import java.awt. Graphics; 

public class hello extends java.applet.Applet 



public void paint( Graphics g ) 

1 g.drawStrinflCHeno from Java!'. 60 , 30 >; 



In this book, you will place your programs into subdirectories of a 
newdirectorycaUed javal-2 (this is optional-you can choose any 
name). That means you'll save the hello . java file as c : Yjaval-Z\ 
hell o\hello; java. 

. NowTOuhaveoeatedhellooava-Thisis to 
\ applet and itcontains the Java «><te that you have writ^Tlie next 
step is to compUe this Java code, into a working applet and see your 
applet at wortApplets have the extensioti .class making the name 
Jyouractualapplet hello.class.Pll shwyou.why applets have the 
extension .class shortly... • ■ 

Setting y RtHf JavaIDIC* 

ryouH use Java itself .to a 

.^Development Kit (JDIQ ... , 4 ...,.„ . 

With previous versions of Java,ypu used to have to gp through a rather 
lengthy andmvolved installatidri process, birt thats lallchanged now- 
ywjusthavetonman.EXEffl^^^ 

■ follow the instructions foir installation. 

The next step is to make sure you can run the JDK from any location 
ta vour conmuter (mduo^the c ^ 

tories, which is where you'll put your Java programs). To do that, make 
*^thePATHstatem£tmy<^^ 

directory of the C drive) includes the JDK BIN and LIB directories (here 
IhavelnstaUedthe JDKin c:\jdkl2-use wh^verpathis appropriate 
to the way you have installed the JDK): 

PATH-C:\WIND0WS;C:\3DIO2\BI»l;C:\30K12\LIB 
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The JDK 1.2 is ready togo. 
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long filenames) * ^ """^ P"**™ ™ be able to handle 
NOTE 

mTeS^ 

^^yousKss^ 

t^materialJater^ 

From Java io to Java 1.1 

ManyreadeiB will be femiliar with. ravp in nn »i 
by looking at the d«CK^ to j^f /8oWew ^ start 




ir* y££* 



iar files .jar (Java Archive) files were introduced in Java 
1 1 and let you package a number (rf files together, zipping them 
to shrink them, so the user can download many files ^t once. 
You can put many applets and the data they need together into 
one .jar file, maldng dcwuoading much faster. The* fflesaie 
analogous to . zi p files except that your browser will download 
them and unrip them orrthe-flyforyoa 
Internationalization Java 1.1 lets you develop locale-specific 
applets, including using Unicode characters, a locale mecha- 
nism, localized message support, locale-sensitive date, time, 
time zone, number handling, and mote. 
Signed applets and digital signatures Java 1.1 can create 
digitally signed Java applications. A digital signature gives your 
u^rs a "path" back to you in case something goes wrong.TJus 
is one of the new security precautions popular on the World 
WideWeb. 

lUmirtemethodinTOcation In Java U,RMIIets Java 
objects have their methods invoked from Java ccderunrung in 
other Java sessions. This is sort of similar to Local Kemote Pro- 
cedure Calls (LRPCs).- - . - t 

-dbjects^ 
leriyourstoieiObjecls 

put stream* Besides allowing you td store comes;?? >f object* 
you serialize, serialization is also the basis of corflmurdcation 



^bet^obj^ 

Reflection In Java LI, reflection lets Java code dmnmeinfor-. 
nation about the methods and constructors of loaded classes 
and make use of those jste^mt*^^ £Pg£$Jfcy < 
Innerclasses. Java i.lmakea it easier toXaeaie adapter r 
classes. An adapter class is a class that implements an interface 
required by an API (Applications Programming Interface) . An 
adapter class "delegates" control back to an endosingmain 
object 

NewJavanativemethodiiitei^aM Native code is code that 
Uvmttenspe<afically«Drapaitic^ . 
interface was introduced to provide a standard programming 



ed in Java 
ppingtfaem 
at once, 
(ether into 
se files are 
(download 

ale-specific 
media- 
^ time, 

n create 
gives your 
tonfrThis 
: World 

lava . 
unhing in ' 
anotePro- 

LI, and it 

^objects 
oication 
jis similar * 

tnineinfbr- 




CtDXS. 
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^ tta »V™ta»fJ.™io,».th«I, 
wie oonadsred obsolete in Java u, and tonus maifed»< 



and] 



vnthA^-^-^^J P^^Hge was extended 



independent of a swwifir ,* a ,£*f^ PiPgrams that are. 

From Java 1.1 to Java 2 

N 0 wlef 8 haveaIookatwhat'8newinJava2 



resource (such as "read" and "write" access to a specified file or 
directory, "connect 0 access to a given host and port, andsoon). 
The policy, specifying which P^^^,. 3 ^ 1 ^^ 
ftomvarious signers/locations, can be initialled fromanexter 
oal configurable policy file. Unless a permission is ^aOy 
granted to code, it cannot access the resource that is guarded 
by that permission. 

Swing (JFO Swing is the part of the Java Foundation Classes 
(JFQ that implements a new set of GUI components with a 
"pluggable" look and feel. Swing is implemented in pure Java, 
and Abased on the JDK 1.1 light-weight UI Framework. The 
pluggable look and feel lets you design a single set of GUI com- 
ponents that can automatically have the look and feel of any 
platform (e.g_, Windows, Solaris, Macintosh). 
Java2D(JFC) TheJava2DAPIisasetofdasses.for 
advanced 2D graphics and imaging. It encoirrpasses line art, 
text,, and images fa a ^^^ammmOA .. 
Accessibility (JFQ TtaoS&JmA^cWafcAH. 
developers will be able to create Java applications thatcan 
interact with assistive technologies such as screen readers, - 

Draff and Drop (JFC) >■ l)rag mflpxop enables data transfer 
across both Java and native applkations, between Javaapphca- 
.. tions, and wtthm a single Java ^phcation, . 

^^Oolle^oiia*^ 
forrepiesenu^andmanip^t^ 

. TOurnorcatoutthmb^ 

r * independent of the details »their jf^*^|gS??^, , Y . 

• Java extensions .' Franiev^AExt^sic^ ^ 
dasses (and airy associated native aide) ^ application devdr 
opers can use to extend the core platform. The extension mecha- 
nism allows the Java Virtual Machine (JVM) to use the extension 
classes in much the same way it uses the system classes. 
JavaBeans enhancements Java 2 provide developers with 
standard means to create more sophisticated JavaBeans compo- 
nents and applications that offer their customers more seam- 
less integration with the rest of their runtime environment, 
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SJS^S^T 0 * ^^n^od framework 
enables aU text-^ditmg components to receive Japanese Chi- 
neae, or Korean text input through standard iirpiX* 

virion identificatioTi -Vendoning- introduces 
where apph'Ss anSapp^ 

^1^™' 35 ^ 35 Custom ^ ^Vpes tteSw a 
remote object to specify the custom socket typeUjat RlSS^if 

such as SSL, can be supported using custom socket types!)^'. . 
API that allows the serialized data of ari object to be sririfi M 

Befia^oyects . A nMice^^L^^ ^ . 
^^ to ?Q»* other obje^ 

so^en^andBupport^ 

Java DM. fattoiOCmMlPmuBnaiteton** 

WeHS ^ J ™ IDL enables dis- 

mowed Wet>enabled Java applications to invoke operations 
tnmsparently on remote network services using theSs^ 
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standard OMG IDL (Object Management Group Interface Defi- 
nition Language) and HOP (Internet Inter-ORB Protocol) 
defined by the Object Management Group. 

JAR enhancements The enhancements include added func- 
tionality for the commancHine JAR tool for creating and updat- 
ing signed JAR files. There are also new standard APIs for 
reading and writing JAR files. 

JNI enhancements The Java Native Interlace (JNI) is a stan- 
dard programming interface for writing Java native methods 
and embedding the Java Virtual Machine into native applica- 
tions. The primary goal is binary compatibility of native method 
libraries across all Java Virtual Machine implementations on a 
given platform. Java 2 extends the Java Native lnterfece to incor- 
porate new features in the Java platform. 

JVMDI A new debugger interface, the Java Virtual Machine, 
now provides low4evel services for debugging. The interface for 
these services is the Java Virtual Machine Debugger Interface 
(JVMDI); ; 

JDBC enhancements Java Database Connectivity (JDBQ is 
a standard SQL database access interface, providing uniform 
. ao^ to awide^iangeo 

- ^ 

: .canbebufltThe Java2 softy^btm^ 
.JDBOODBC bridge. . 



These corteepte^ ready 



'vtocon^ilethc 



Compiling the Hello App^ ^ 

Now that you have installed fteJDRanB^ 

file ready to gb) ; you ofeatel^ see it run, To do this, 

change. to the c : \ javal-2\hel 1 6 directory now (or wherever you have 
saved the he! 1 o o ava file); this is how the DOS prompt should look: 

c:\javal-2\hel lo> 
Next, type this to create your applet 

c:\javal-2\hel 1o>javac helToljava 
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DOSoonunand SK^SSjSi ^ type the 

** both hel 1 ooava ^ heit ^s^I 00 ^' ^ 
hello, class, yoiuaiXis^J^!^ 6 ^ ***** 
What have you really *me? * 8 °~ but *» that mean? . 



Understanding Java 

Lefs take the 



use. In this case, that means ZnafllT 80metiu f« 8 computercan 
named hel ^c^s^lT?l^?^ top ^ thefi te 

net-it doesn't matter wh*f i*Z^Jl J? " 80 POP"^ on the Inter- ■ 
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that makes up your applet with JVM's doss loader and then runs the 
applet 

Running the Hello Applet 

Tb see hel lo . cl ass, your firet applet, running, youTI need a Web page 
to place it in. Use your editor again to create a new file, hello, htm, 
which will be your Web page, written in the language of Web pages, 
Hypertext Markup Language (HTML) (well review HTML in a minute) . 
Enter the following text into h e 1 1 o . h tm and save it in the same direc- 
tory as the hell o . cl ass file: 
<html> 

<!- Web page written for the Sun Applet Viewer> 
<head> 

<t1tle>hello</title> 
. </head> 

^body> 
<hr> 

<app1et v v ' 

'./^/heig^OO^.^;,.;.. ;\...., .... - 
\'^applet>"*' * : * 

</body> . hi .'-■/ ; V.*-; . . ~ ' 

Wowybucan run the heflo applet by simpty viewing this new Web 
page, h ell o : htm. To dp that, use the Applet Viewer that comes with the 
JDK 12. To tise the Applet Viewer; go back tp the hel 1 o subdirectory 
and type the Mowing: 

c:\javal-2\hello>appletviewer hello. htm ■ 

Again, capitalization is very important here— make sure your capitaliza- 
tion matches the exact spelling of the Web page name. When you've done 



to your applet In this way, well be able to draw the text string, " 
fiom Java!", in the applet's window. 




much like the C/C++ #lndude statement 



Nest, add these lines to hell o. java: 
Import java.awt.Graphics; 
public class hello extends java.applet.Applet 



( 
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graphicsClass screen; •<*.••'. 

YouTI see how to actually create a class *v»n . M 

a raphi csCl ass i* 3~ ^ 60011 (<?reating a class like 
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youll see how to create objects of that class. What's important to remem- 
ber is this : the object itself is what holds the data you want to work with; 
the class itself holds no data but just describes how the object should be 
setup. . 

Object-oriented programming at root is nothing more than a way of 
grouping functions and the data they work on together to make your pro- 
gram less cluttered. Youll see more about object-oriented programming 
throughout this book, including how to create a class, how to aeate an 
object of that class, and how to reach the functions and data in that object 
when you want to. 

That completes the minkjverview of classes and objects. As you can 
see, a class is just a programming construct that groups together, or 
encapsulates, functions and data, and an object may be thought of as 
a variable of that class's type, as the object screen is to the class 
screenclass. 

As if turns out, Java comes complete with several libraries of prede- 
fined classes, which save you a great deal of work. Thioughout.this book, 
w$ will examine these predefined and very useful Java classes. Using 
these predefined classes, well create objects needed to handle buttons, 
text fields, ; scroll bars, and much more. 




Learning about Java Packages 



These class libraries are called packages in Java, and one such library is 
^called j ava i awt <where awt stands for Abstract Window Toolkit). This 
-Ubraiy holds the Graphi cs class/which will handle the graphics work 
you undertake. So this line in the hello, j ava file: 

. . -import Java, awt .Graphics ; . .. 

actually means that you want to include the Java Graphics class and 
. make use of it in your program. In a minute, you will use an object of the 
. Graphics class for your graphics output 

You've added support for graphics handling by including the j ava 
. awt . Graphi cs class (and in Java, displaying the text string "Hello 
; from Java!" is considered graphics handling). Next, it's time to set up 
your hello applet itself. To do so, define a new class named he! 1 o. 
This is the standard way of setting up an applet in Java, and in fact, the 
applet itself has the file extension . cl ass. That's because each class 
defined in a ijava file ends up being exported to a .class file, where 
you can make use of it Youll learn more details about this soon. 
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from Zfti pT^ ^ <«de an applet dass need, 

fem^HtA For sample, we'd need to interact with the Web biow^ 
n*em a secbon of screen, initialize the appropriate Java pa^aTTd 
much more. It tons out that all tJmtf^ctionityisaliJStiXrle 
JavaApp1etda S s,wnicfalsi)moftliejava. a pple1S 

^SSo^ 

Understanding Java Inheritance 

hSwi^Ttf e . java ' a PP le t. Applet class by denmn* the 

th] Sl*?^ 60,3 java ' Witt .Applet. VtoiL 

JSSS P^** 6 java.applet.AppletdassvrfthootSr 

daTbtl^i? ^ ^ 0311 *»* want to^s 

class by adding code to your derived class hello. 

^^fatlus way, aden^dassuiherite 

frZ!tTT^ Ctbe8ame chassis, they add* different : 

^tothebas^endingupas^ 

use the keyword doss to indicate that you are defining a new cS 
import java.awt. Graphics; 

public class hello extends java.applet .Applet 




pai ntC) method Bke this: 
? " import java.awt-Graplrfcs; 

• \ public class hello extends java.applet.Applet 
( public void paintC Graphics g )C 




^^^^^^^ 

parts are called aclases members. 



Building the first Java Ex amples 

What Are Java Access Modifiers? 

dedared publ i c, pn vatre, or protected. If tb^ aie debated publ ic 

a JSi"^ ^ t » e of ^P* 1 n * O method. When you call 
a method, you can ^parameters to it, and it can return data to w,u. 
^<*f< pa ; «0 has no return value; which you indicate with the 
return type port Other return types are i nt for an integer retumX 

^j^l^^y 0 ""^ that the paint() method is automati- 
oallypassedonepaiameter-wobjertoftbe^^ 

import java.awt. Graphics; . ■ . 

J" b V5 «t««ls jav a .appUt.AppVet - >■ - 

public void palntCGraphics 9) 



This Graphics object represents the physical oasplay of the applet 
^ you can use the built-in methods of this object-^ 2drlw- 

« WLlne0 ' dra ^ a H).andothen^diawnonto 
Jh™? you want to place thestring "Hello from Java!" on 

the screen, and you can do that with the d ravySt ri ng ( ) method. 
How do you reach the methods of an object like the G raDhi cs obi«* 

aSSCT ? *»^*^tr!ngO methoJto W a 
S!S ° n ^ e 805611 ^ ^ handled like any other type of gnjphics 

£ZS^ "^"J*" *' k 18 ^ on ^ rathertL 
pnnted just as ;t)u would drawarectangle or circle). Svmfv three 

play, and the (x, y) location of that string's lower-left comer (ciuled the 



starting point of the string's baseline) in pixels on the screen, P^d j" 
^ro inteSrvalues. As shown in Figure 1.3, you can draw your string at the 
^Sn1o?30), where (0, 0) is the upper-left comer of the applet's 
display. 



mwnllnate system !na|avaprogramUsetup^*eortglnp.©^e 
doTnwarfs; this fact will be important throughout the book. 
^o^. where you start at the upper4eft and work your way to the right and 
screen pixels. 
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FIGURE 1*3: Drawing a string at (60,30) 

This means that you addacall to thedrawStringO method this^ 

import java.awt-Graphics; 

public class hello extends java. applet. Applet 

{ public void palntC Graphics g ) 

1 g.drawStringC -Hello from 3ava!\ 60, 30 ); 
} 

• Note that Java use* the sam^ 

code statement is finished: it ends the statement with a semicolon (;) . 



TIP 

tngeneraUavaadheresverys^ 
C++, you already know a great deatof Java. 
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You have completed the code necessary for this applet, which is also 
to say you have completed the code for the new class, hello. When the 
Java compiler creates hello, cl ass, the entire specification of the new 
dass will be in that file. TWs is the actual binary file that you upload to 
your Internet Service Provider so that it may be included in your Web 
page. A Java-enabled Web browser takes this class specification and cre- 
ates an object of that class and then gives it control to display itself and, 
if applicable, handle user input - 

But how? You have not yet completed the dissection of the first example; 
all you have done so for isto trace the development of hello, java into 
he! 1 o . cl ass. How did you get the applet to be displayed in the Applet 
Viewer? 

Understanding the Applet's Web Page 

The Applet Viewer took the hello, clas s applet and displayed it in a 
Web page, as shown in Figure 1.4. 



Web page 



Applet ; . 



FIGURE 1.4: Displaying an applet in a Web page 

How did it get there? You created a Web page for your applet and then 
opened that Web page in the Applet Viewer, which then displayed your 
applet Tliat Web page looks like this; 

<htm1> 

<l- Web page written for the Sun Applet Viewer> 

<head> . 
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<tit1e>hel1o</t1t1e> 
</head> 

<body> 
<hr> . 

<app1et 

code-hello. class 

width-200 

height«200> 

</app1et> 

<hr> 

</body> 

</htm1> 

Web pages aie written in HTML (Hypertext Markup Language^ 
Because applets appear in Web pages, we wiU take the time to briefly 
work through the above page to make sure you know what s going on. If 
you're familiar with HTML, you can skip much of this review, but you 
should take a look at how to use the <appl et> tag to embed applets in 
Web pages. 

Connecting Java and HTML 

Let* s take apart the Web page you created for the applet now, starting 
with the <html> tag: 

<html> 



Instructions in . html pages are placed into tags surrounded by angle 
brackets: < and >. The tags hold directions to the Web browser andare 
notdisplayedonthescieen.Here,to^ 
browser that this .html file is written in HTML. 

Next comes a comment Comments in .html pages are written using 
thelsymbollikethis: <! This is a ctwment.Xuidicatethattiusis 
a Web page written so that we can use the Sun Applet Viewer, like this: 

<html> 

<ir Web page written for the Sun Applet V1e«er> 
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end header ta^SXn? SSSZF*™ ^ ^ "'Ending 
such as <h ead> and X/h eadT SSSSL'* J?? ^P^^thi* 
text and iznages). In ftSSJX ^JX* *! d <'«nter> to center 

<html> 

<»- Web page written for the Sun Applet Viewer> 
<head> 

<tit1e>hello</title> 
</head> 



<htm1> 



<l- Web page written for the Sun 



Applet Viewer> 



<head> 

<title>hello</title> 
</head> 

<body> 

<hr> 



Web pa* written for the Sun App let Vi^ r > 
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<head> 

<titlOhello</title> 
</head> 

<body> 
<hr> . 
<applet 

code«hello. class 
wi<Kh*200 
height«200> 
</applet> 




Youcanalsousethejava;app1et.Appl€t,res1zc()raethodIn^ 
code to request thatthe Web browser resize applets. 



The <appl et> tag is important, so let* 8 take a closer look at it now. 
Here's how the <appl et> tag works in general (the items in square 
brackets are optional, and the others are required): 

<APPLET> . . • ' twv - r 

[AllQi - LEFT or RIGHT or TOP or TEXTTOP or MIDDLE or 

ABSMIDDLE or BASELINE or BOTTOM or ABSBCTTOM] 
[ALT - AUemateText] 
CODE * AppletName. class 
[CODEBASE - URL of .class file] 
HEIGHT • AppletPixelsHeight 
[HSPACE - PixelSpaceToLeftOfApplet] 
[NAME * Applet InstanceName] 
[VSPACE * PixelSpaceAboveApplet] 
WIDTU - Applet PI xelsWidth 

[<PARAM NAME.- Parameterl VALUE « VALUEl] 
[<PARAM NAME e Parameter2 VALUE • VALUE2] 



</APPLET> 
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TIP 



Indicate to the Web browser here how much space you 11 need for ™ , r 

etas to applets with the PARAM keyword like this: <applet> PARAM 
today - -friday </applet>. Passing paraSsto^^ 
n™S U ^? stomize I 0 ™ ***** to fit different Web pages because 
you can read the parameters from inside an applet and nateuseof^. 




TIP 

pass the name of . j ar files as parameters. You'll learn more about this lateron! 

hJ^f 1 ^- ^f^^PPort Java. In practice, this means that those 
browse 1 B,u S t I gr 1 o re th e <appl e t>tag. This, in turn. means that youS, 
pboetett between the ^pleOandVapple^tag^^b^dT 
Played in nonJava biowse* (and not in Java^rmbled browsers), KkeUds- 
<applet code°hello> 

Zut* So^r d0€S ^ $upport ,ava - so .^ «* 

</applet> 

^ done in this temporary page. Finish off the Web pa^tinL 
</body> and </html> tags as follows: w 
<htm1> 

<!- Web page written for the Sun Applet V1ewer> 
<head> 

<tit7e>hello<ytitle> 
<7head> 

<body> 
<hr> 

Capplet 

code-hello. class . ■ 
wldth-200 



m 
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height-200> 

</applet> 

<hr> 

</body> 

</html> 

This completes our Gist example-you've had a glimpse into the 
process of creating and running an applet It was as quick and easy as 
that-you created and ran your first applet 

What's Next? 

In this chapter, the example applet demonstrated the easiest way to get 
an applet to work. Lefs continue on to get a better idea of how youH be 
working with Java throughout the book as you give your applet more 
power in Chapter 2. 
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Abstract 

This paper describes a new public-key cryptosystem based 
on the hardness of computing higher residues modulo a com- 
posite RS A Integer. We introduce two versions of our sc heme, 
one deterministic and the other probabilistic The determin- 
istic version is practically oriented: encryption amounts to 
a single exponentiation w-r.t. a modulus with at least 768 
bits and a 160-bit exponent. Decryption can be suitably op- 
timized so as to become less demanding than a couple RS A 
decryptions. Although slower than RS A, the new scheme is 
still reasonably competitive and has several specific appli- 
cations. The probabilistic version exhibits an homomorphic 
encryption scheme whose expansion rate is much better than 
previously proposed such systems. Furthermore, it has se- 
mantic security, relative to the hardness of computing higher 
residues for suitable moduli. 

1 Introduction 

It is striking to observe that two decades after the discovery 
of public-key cryptography, the cryptographer's toolbox still 
contains very few asymmetric encryption schemes. Conse- 
quently the search for new public-key mechanisms remains 
a major challenge. The quest appears sometimes hopeless 
as new schemes are immediately broken or, if they survive, 
are compared with RSA, which is obviously elegant, simple 
and efficient. 

Similar investigations have been relatively successful in 
the related setting of identification, where a user attempts 
to convince another entity of his identity by means of an on- 
line communkation. For example, there have been several 
attempts to build identification protocols based on simple 
operations (see [33, 35, 3$, 36)). Although the question of 
devising new public-key cryptosystems appears much more 
difficult (since it deals with trapdoor functions rather than 
simple one-way functions), we feel that research in this di- 
rection is still in order: simple yet efficient constructions 
may have been overlooked. 

The scheme that we propose in the present paper uses ' 
an RSA integer n which is a product of two primes p and q t 

Permission to make digital or hard copies of all or part of this work for 
personal or classroom use is granted without fee provided that copies 
are not made or distributed for profit or commercial advantage and that 
copies bear this notice and the full citation on the first page. To copy 
rthmvttc to wpubtoh. to post en servers or to redistribute to lists, 
requires prior specific perrnission and/or a fee. 
5th Conference on Computer & Communications Security 
San Francisco CA USA 

Copyright ACM 1993 MI1MOT40SAl~SSJ0O 59 



as usuaL However, it is quite different from RSA in many 
respects: 

1. it encrypts messages by exponentiating them with re- 
spect to a fixed base rather than by raising them to a 
ffaced power 

2. It uses a different "trapdoor* for decryption 

3. its strength is not directly related to the strength of 
RSA 

4. it exhibits further ^algebraic* properties that may prove 
useful in some applications, 

We briefly comment on those deferences. The first one may 
offer a competitive advantage in environments where a large 
amount of memory is available: such environments allow 
impressive speed-ups in exponentiations that do not have 
analogous counterparts in RSA-like operations. The second 
is of obvious interest in view of the fact quoted above that 
there are very few public-key cryptosystems available. With- 
out going into fcprfmtral details at this point, let us simply 
mention that the new trapdoor is obtained by injecting small 
prime factors in p— 1 ando;— 1. In order to understand what 
the third difference is, we note that, if the modulus n can 
be factored, then both RSA and the proposed cryptosystem 
are broken. However, it is an open problem whether or not 
RSA is "equivalent 0 to factoring, which would mean that 
breaking RSA allows to factor. For this reason, the hypoth- 
esis that RSA is secure has become an assumption of its 
own, formally stronger than factoring. Our cryptosystem is 
related to another hypothesis, also formally stronger than 
factoring and known as the higher residuosity assumption. 
This may help to understand how these various hypotheses 
are related. Finally, we will explain the al gebra i c property 
of our scheme (called the homomorphic property) by means 
of an example: suppose that one wishes to withdraw a small 
amount u from the balance m of some account; assume fur- 
ther that the balance is given in encrypted form E(in) and 
that the clerk performing the operation does not have access 
to decryption. The cryptosystem that we propose simply 
solves the problem by computing E(m)/B(u) mod n, which 
turns out to be the encryption of the new balance ro — u 

The ability to perform algebraic operations such as addi- 
tions or substructions by playing only with the cryptograms 
has potential applications is several contexts. We quote a 
few: 

1. in election schemes, it provides a tool to obtain the 
tally without decrypting the individual votes (see {4]) 



2. in the area cf vatermazHng, it allows to add a made 
to previously enoypted data (as explained ia [2$D- 

Stfll, in these contexts, it is often needed to encrypt data 
taken from a small set 5 (eg. 0/1 votes) and it is well known 
that detenninisticayptoVystems, sach as RS A, finl here in 
order to decrypt B(a) t one can simply compare the cipher* 
text with the encryptions of aU members of S and thus find 
the correct value of o. In order to overcome the difficulty, 
one has to use probabilistic encryption, where each plaintext 
has many corres pond i ng dphertexts, depending on some ad- 
ditional random parameter chosen at encryption time. Such 
a scheme should make it impossible to distinguish encryp- 
tions of distinct values, even if these are restricted to range 
over a set with osjy two elements. This very strong require- 
ment has been termed semantic security Q12J). As a farther 
difference with RSA, the cryptosystem introduced in this 
paper, has a very natural probabilistic version, with proven 
semantic security. 

The probabilistic homomorphic encryption schemes pro- 
posed so far suffer from a serious drawback: they have very 
poor bandwith. typically, they need something like one kilo- 
bit to encrypt just a tew bits, which is a quite severe expan- 
sion rate. Tab may be acceptable for election schemes but 
definitely hampers other applications. The main achieve- 
ment of the present paper is to reach a significant band- 
with, while keeping the other properties, inrinrrtng semantic 
security. 

Before we turn to the more technical developments of our 
paper, it is in order to compare it with earlier work: it is in- 
deed the case that the question of finding trapdoors for the 
discrete logarithm problem has been the subject of many pa- 
pers. At this point, it is fair to mention that the probabilistic 
cryptosystem that we propose is actually quite close to the 
most general case of the homomorphic e ncry ption schemes 
introduced by Beoaloh in his Ph-D thesis [41. Stfll, both in 
this thesis and in the related work ((5, 6, 7j), the security 
and potential applications are only investigated in a setting 
where the bandwith remains smalL A more recent paper 
by Park and Won (see [24))describes a related probabilistic 
cryptosystem using a trapdoor based on injecting a single 
power of a small odd integer into p - 1 or q — 1 and proves 
its security with respect to an ad hoc statement. Thus, our 
paper oners the first thorough discussion of the security of 
a probabilistic homomorphic encryption scheme with signif- 
icant bandwith. After the completion of the present work, 
we hare been informed that another homomorphic proba- 
bflistfc encryption scheme, using moduli n of the form p?9, 
where p and q are primes, had been found by Okamoto and 
Ucbiyama (see [22]), achieving an expansion rate similar to 
ours. Finally, it should be emphasized that the determinis- 
tic version of our scheme is not simply a twist that fixes the 
random string in the probabilistic version: considering its 
practicality, we believe that, even if it is not intended to be 
a direct competitor to RSA, it enters the very limited list of 
efficient public-key cryptosystems. 

The paper is organized as follows: in the next two sec-* 
tioos, we successively describe the deterniinistic and the 
r^bablHstic version of our scheme, the former with a prac- 
tical approach, the latter in a more complexity-theoretic 
spirit We then discuss applications and end up with a chal- 
lenge for the research community. 

2 The deterministic version 

As was just mentioned, our approach to the deterministic 
scheme is practically oriented: we discuss system set-up 



and key-generation, encryption and decryption, with per* 
fannanrps in mind. We ate carry on a security analysis at 
the informal level and we derive <wiH " >ft| sugested parame- 
ters. 

2JL System set-up and key generation 

The scheme that we propose in the present paper can be 
described as follows: let v be a straarefree odd B-smooth 
integer, where B is small integer and let n a pq be an RSA 
modulus such that a divides ${n) and is prime to #(n)/o\ 
Typically, we think of B as being a 10 bit integer and we 
consider n to be at least 768 bits long. Let g be an element 
whose multipucative order modulo n is a targe multiple of 
<r. Publish n, g and keep p, q and optionally a secret. A 
message m smaller than <r is encrypted by g m mod n; de- 
cryption is performed using the prime (actors of c as wul be 
seen in the next subsection. 

Generation of the modulus appears rather straightfor- 
ward: pick a family p» of Jk small odd distinct primes, with 
* even. Set ti = fll^F*. v = ITjU+iP* and a = tu; = 

IXLiPf* Pick two large primes o and 6 such that both 
p = 2au -f 1 and q = Tbv + 1 are prime and let n « pq* 

However, this generation is lengthy especially when the 
sue of the modulus grows: o has to be chosen in the appro* 
priate range and tested for prim&tity as well asps 2au + 1 
untO both tests succeed simultaneously. This might be a bit 
time-consuming. Instead, we suggest to generate a, 6, u and 
v first (independently of any primafity requirements on p 
and q) and use a couple of 24-bit "tuning primes*' p* and 4 
(not used in the encryption process) such that p = 2oup'+l 
and q = 7bvq* + 1 are primes. To avoid interferences with 
the encryption mechanics, we recommend to make sure that 
gcd(pY,0") = 1 and jf £ g*. In practice, such an approach 
is only 9% dower than equivalent-size RSA key-generation. 

To select g, one can choose it at random and check 
whether or it has order £(n)/4. The main point is to ensure 
that £ is not a p»-th power, for each i < k by testing that 

c »< ' 1 mod n. The success probability is : 

* 1 * 1 

v = JJ(1 ™ ~)t whose logarithm is : In(ir) rr 

If the pi& are the first Jfe primes, this in turn can be estimated 
as -Inlnfc and results In the quite acceptable overall prob- 
ability of * 2 1/ In fc. Another method consists in choosing, 
for each index i < h t a random o ( until it is not a p.-th 
power. With overnrhelming probability g = Flfoi 9*1™ ^ 
order > #n)/4. 

2J2 Encryption and Decryption 

Encryption consists in a single modular exponentiation: a 
mreqagp m smaller that a is encrypted by g m mod*. Note 
that it does not require knowledge of <r, A lower bound 
(preferably a power of two) is enough but it is unclear how 
important for the security of the scheme is keeping a se- 
cret. However, if one chooses to keep c secret, necessary 
precauti o ns (similar to these applied to Rabin's scheme (31) 
or Shamir's RSA for paranoids (34]) should be enforced for 
not being used as an oracle 1 . 

'For example, an attacker having access to a decryption box can 
decrypt a™ mod n (or some m > v and set m mod o\ Tab disclose* 
(by iobteaction) a multiple of *■ and e can then bo found by a few re- 



60 



Also, there is actually no reason why the p& should be 
prime. Everything gpes through, nrotats? mutandis, as soon 
88 the pi8 axe mutually prime. Thus, for example, they can 
be chosen as prime powers, which is a way to increase the 
variability of the scheme. 

Decryption is based on the Chinese remainder theorem. 
Let in, 1 < i £ i, be tie prime fectars of ov Hie algorithm 
computes the value m* of m modulo each jh and gets the 
result by Chinese remaindering, following an Idea which goes 
back to the Pohlig-HsUman paper [27]. In order to find rm t 
given the dphertext c«j" mod n, the^alp rithm computes 

Ci = c & mod n, tsrhich is exactly p n mod n. This fol- 
lows from the following easy computations, where yg stands 



for 2=at : 



c; = c '< 



*g w modn 



By m^apaHng this result with all possible powers j « , it 
finds oat the correct value of m,\ In other words, one loops 

for j ss 0 to pi — 1 until a • g w modn. 

The cieartext m can therefore be computed by the fol- 
lowing procedure : 

forial to fc 
{ 

let c<s=c* o)/w modn 

lor j = 0 to p< — 1 
{if Ci = o*««l/« mod n I*t rm-j} 
} 

The basic operation used by this (non-optimized) algo- 
rithm is a modular exponentiation of complexity log*(n), 
repeated less than : 

*Pfe < log(n) p* S£ log(n) Ar log(*) < log 3 ^) log iog(n) 

times. Decryption therefore takes log*(n)loglog(n) bit op- 
erations* 

This is clearly worse than the log*(n) complexity of BSA 
but encryption can be optimised if a table stores ail possi- 
ble values of t[Ui\ » o**?\ for 1 < i < k and I <j < U 
the value m, of the cieartext m modulo p« is found by ta* 

ble look-up, once c w mod n has been computed. It is not 
really necessary to store all a »* . Any hash function that 

distinguishes g from g " f for j £ f will do and, in 
practical terms, a few bytes wul be enough, for example ap- 
proorimately 2ip»| bits from each t[t, j]. It is even possible to 

use hash functions that do not discriminate values of g « : 
the proper one is spotted by considering, by table look-up 

petted trials and gcd*. Tb provost such an action, the decryption box 
cannot only re-encrypt and check against the dphertext received, a* 
thb aOm a starch ty dichotomy. It should first check that the diar~ 
text b to the appropriate range, e.g. < 2* with < m # la sncijrpl it 
and then check that It matches tip with the original dphertext before 
letting anything out. 



hashes of g « f for I » 1, 2, • • • until there is no ambi- 
guity. Hits can be very efficiently implemented by storing 
hash values is increasing order wx.t- t and one single bit 
might be enough. 

23 A toy example 

• key generation tat k » 6 
p 21211 =2x101x3x5x7 + 1, 

9=928643 = 2x191x11x13x17 + 1, 

n s 21211 x 928643 « 19697446673 and g m 131 yield the 
table: 





t'=i 


i = 2" 


i = S 


tzr 


7=5" 






0001 


0001 


0001 


0001 


0001 


0001 




1966 


6544 


1967 




"TOT 


0372 


J = 2 


9560 


3339 


4968" 


7876 


4792 


7767 


7=tt 




9400 


• Tils' 


"57ST 


0262 


3397 






5479 


6701 


79M 


0136 


0702 








6488 


8651 


6291 


4586 


i= 6 






2782 


4691 


6677 


8136 










9489 


1690 


3902 










8537 


6878 


6930 










2312 


Vsti' 


6399 










7707 


7180 


6592 












8291 


9771 












0678 


0609 














7337 


j = 14 












6892 














3370 


i = i6 












3489 



where entry {£,7} contains gM*Np* mod n mod 10000* 
• encryption of m = 202 



c = o m mod n = 131*° mod 19697446673 = 519690214 

• decryption 
by exponentiation, we retrieve ; 

c n mod n mod 10000 = 1966 
*2l 

c 73 modn mod 10000 = 3339 
C» modn mod 10000 =3 2732 
c m modn mod 10000 *■ 7994 
e ps mod n mod 10000 = 1890 
c »a mod n mod 10000 — 3370 
wherefrom, by table lookup : 

mmod3«tafal«(1966) =1 
m mod 5 » table (3339) = 2 
m mod 7 = table (2782) =6 
m mod 11 « t«ble(7994) = 4 

m mod 13 » table (1890) = 7 
m mod 17 = table (3370) =5 15 

and by Chinese remaindering : m = 202. 
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2*4 Suggested pajamelefs and security analysis 

We suggest to take a > 2**° and we consider |n{ = 768 bits 
as a TfttniTntpn ate for the modulus. 

If the fectoraatios of n is found, then a and o become 
known as well as Kn)< The scheme Is therefore broken. 
However, the scheme does not appear to be provabiy equiv- 
alent to factoring. Rathe, it b related -to the question of 
having oracles that decide whether or not a random num- 
ber x is a j>i-th power modulo n, far i — 1, This is 
known as the higher residuosity problem and is currently 
considered unfeasible. Formal equivalence of this problem 
and the probabilistic versto onr encryption scheme will 
be proved in the next session. Considering the bask deter- 
ministic version, we have no formal proof but we haven't 
found any plausible line of attack either. Abo, the efficient 
factoring methods such as the quadratic sieve (QS) or the 
number field sieve (NFS) do not appear to take any advan- 
tage from the side uofonnation that u (resp. o) divides?— 1 
(resp. g — 1). The same is true of simpler methods like 
Pollard's p—1 since we have ensured that neither p— 1 nor 
q — 1 is smooth. Finally, elliptic curve weaponry (18) will 
not pull-out factors of n in the range considered. Note that 
the requested size of n (768 bits or more) makes factoring n 
a very hard task anyway. 

We now turn the size of o\ In order to avoid the com- 
putation of discrete logarithms by the baby step-giant step 
method, we have to make <r large enough. As already stated, 
2 160 is a minimum. This can be achieved for example by 
making a a permutation of the first 30 odd primes, which 
yields a cr 2 6 °** 5 . Alternatively, one one can choose a se- 
quence of 16 primes with 10 bits. Since there are 75 such 
primes, this leads to a ^ 53-bit entropy. Adding prime pow- 
ers, as stated above, will further increase these figures. 

There is a further difficulty, when <r is known. Note that 

hence 4o5 differs from J only by e = -E±JzI. The nu- 
merator is of size |n(/2, hence, if it does not e x c eed the 
denominator by a fairly large number of bits, the value of 
ab is basically known and decryption can be performed. 

When the exact splitting of the factors of <r into u and 
v are known as well, the previous analysis can be pushed 
further. Reducing the relation n = (2ou+l)(2ov+l) modulo 
ti, we find that n * Tbv + 1 mod u and we can calculate 
d = b mod u. Similarly, we learn e = a mod v. We let 
a = rv + c and b « su Hh d, with r, s unknown and, using 
the met that c * ut>, we obtain: 

n = (2nw + 2cu + l)(2suv + 2du + 1) » 

4w a + 2o\fi2d»+ 1) +a(2cu+ 1)J + (2eu+ l)(2dv+ 1) 
which is of the form 

n = 4w* + 2e(atr +fis) + 7 

with known or, fi and 7. Reducing modulo o* t this provides 
the value 9 of ar+/fc mod o\ At this point, our analysis be- 
comes quite technical and the reader may skip the following 
and jump to the conclusion that n » <r*. 

For the interested reader, we note that the pair (r, *) lies 
in the two-dimensional lattice L defined by 

L =5 {(x,y)|ox +(fyzz6 mod <r} 



Thb lattice has determinant o\ Abo, it b easily seen that 
a aiui0 are bounded by 2? and 7 t>y 4?** From thb we get 



rs<^j^rs + r+e + li 



(r+l)(s + l) 



Thus, the pair (r,«) b very dose to the boundary of the 
curve C with equation xy = jgy. More precisely, the dis- 
tance between the pair (r,s) and the curve does not exceed 
Thb defines a geometric area A that includes (r,s). 
Now, key generation usually induces constraints that limit 
the possible range of the parameters. For thb reason, it is 
appropriate to replace C by the fine * + y = ^ in order 
to estimate the size of A. Thb leads to an approximation 
which b O(^). The number of lattice points from L in 
thb area is, in turn, measured by the ratio between the size 
of A and the determinant, which b It b safe to ensure 
that thb set b beyond exhaustive search, which we express 
byn»* 4 . 

Note that the ratio |nj/|<r| b the «*p**i«?nn rate of the 
encryption, where (n| denotes, as usual, the size of n in bits. 
It b of course desirable to make thb rate as low as possible. 
On the other hand, as a consequence of the above remarks, 
we see that ^ — |<rj should be large. Asymptotically, thb 
b achieved as soon as we fix an expansion rate which b 
> 4. For real-size parameters, we suggest to respect the 
heuristic bound ^ — 9 > 128, which b consistent with 
our minimal parameters. Larger parameters allow a slightly 
better expansion rate. 

2*5 Performances 

Despite its expansion rate, the new cryptosystem b quite ef- 
ficient: encryption requires the elevation of a constant 768- 
bit somber to a 160-bit power. Several batch ([21, 23]) and 
pre-processing ([2]) techniques can speed-up such computa- 
tions, which might be a small advantage over RSA. 

Decryption b slightly more awkward since k exponenti- 
ations are needed. But thb number can be reduced in a few 



Firstly, while computing c^ 10 '* modn for each i, it 
b possible to first store d « c 4 ** modn and raise c' to 
the successive powers a/pi so that (besides the first one), 
the remaining exponentiations involve 160-bit powers. One 
can further, in the square-and-multipry algorithm, share 
the "square* part of the various exponentiations. A care- 
ful bookkeeping of the number of modular multiplications 
obtained by setting |n| = 76ft and choosing sixteen 10-bit 
primes pi, shows that the total number of modnlar multi- 
plications decreases to 2352: 912 for the computation of c* 
and 1440 Ear the rest Actually, the "multiply" part can be 
somehow amortized as well: we refer to [21] for a proper de- 
scription of such an optimized exponentiation strategy. The 
resulting computing load b less than what b needed for a 
couple of USA decryptions with a similar modulus. 

Unfortunately, there b a drawback in reducing the value 
of k: in the 30-pxime variant it b necessary to store 1716 
different tfi t j\ hash values. Hashing on two bytes seems 
enough and results in an overall memory requirement of four 
kilobytes. In the 16-prime variant, hash values of 3 bytes 
are necessary and the table size becomes s 100 kilobytes. 
As observed at the end of section 2.2, the hash table can 
be drastically reduced at the cost of a minute computation 
overhead. 

Another speed-up can be obtained by separately per- 
forming decryption modulo p and q so as to take advantage 
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of smaller operand sizes. This alone, divides the decryption 
waxkfsctor by four. 

Finally, decryption is inherently parallel and naturally 
adapted to array processors since each rm can be computed 
independently of afl the others. 

2*6 Implementation 

The new scheme (768-bit n, k = 30) was actually imple- 
mented on a 6*HC05-based ST16GF54 smart-card (4,096 
EEPROM bytes* 16,384 BOM bytes and 352 RAM bytes). 
The public key is only 96-byte long and as in most smart- 
card implementations, n*s storage b avoided by a frnnmand 
that re-computes the modulus from its factors upon request 
(re-computation and transmission take 10 ms). For further 
space optimization 9*s first 91 bytes are the byte-reversed 
binary complement ofn*s last $1 bytes. Decryption (a 4,119- 
byte routine) takes 3,912 ms. Benchmarks were done with 
a 5 MHz oscillator and ISO 7816-3 T=0 transmission at 
115,200 bauds. 

3 The probabilistic version 
3J The setting 

We now turn to the probabilistic version of the scheme. As 
already explained, we adopt a more complexity-oriented ap- 
proach and, for example, we view B as bounded by a polyno- 
mial in logn. The probabilistic version replaces the cipher- 
text g m modn by c = x*o m mod n, where x is chosen at 
random among positive integers < n. Decryption remains 
identical This is due to the fact that the effect of multi- 
plying by is cancelled by raising the dphertext to the 
various powers as performed by the decryption algo- 
rithm. Note that this version requires a to be public 

The resulting scheme is homomarphic, which means that 
E(m + m' mod c) = E{m)E(m') mod n. Probabilistic ho- 
momorphic encryption has received a lot of applications, 
both practically and theoretically oriented. To name a few, 
we quote the early work of Beaaloh on election schemes ([41) 
and the area of zero-knowledge proofe for NP (see (13, 3j). 
Known such schemes are the Quadratic ResiduosUy schemes 
of Goldwasser and Mscati ([12]) which encrypts only one bit 
and its extensions to higher residues modulo a single, prime 
(see (4j), which encrypts a few bits. As already explained 
in section 1, these schemes suffer from a serious drawback 
a complexity theoretic analysis has to view the deartext as 
logarithmic in the size of of dphertext. In other words, 
the expansion rate, Le. the ratio between the length of 
the dphertext and the length of the deartext is huge, in 
our proposal, this ratio is exactly £}. Note that that our 
assumption that a is B-smooth, for some small B, does 
not predude a linear ratio. The maximum size of a is 
L^p<B 1°6P» where p ranges over primes and it is known 
to* 6 ( B ) = Ep<a - B * Thus, even if Bis logarithmic 
in n, there are enough primes to make (oj a linear propor- 
tion of |n|. This is a definite improvement over previous 
homomorphic schemes. Note however that, following the 
comments In section 2.4, it is sale to take < 1/4. 

3*2 A complexity theoretic approach 

We already observed that the security of our proposal is re- 
lated to the question of d"rtfog i mffhmg higher residues mod- 
ulo n, that is integers of the form x^modn, when p is a 



prime divisor of ${n). In the rest of this section, we want 
to clarify this relationship in the asymptotic setting of com- 
plexity theory. In view of the remarks just made, we find 
ft convenient to assume that the ratio j^j has a fixed value 
a < 1/4. We also fix a polynomial B in logn. The parame- 
ters which are of interest to us are pairs (n,o-) such that c is 
squarefree, odd and B-smootb, n is a product of two primes 
P> 3> * is a divisor of 4(*i) prime to 4(n)/? and j|J — o% We 
call any integer n that appears as first coordinate of such 
a pair (B,a)-dense. Distinguishing higher residues is usu- 
ally considered difficult (see [4]). We conjecture that this 
remains time when n varies over (B t a)-dense integers. To- 
wards a more precise statement, let Bp(y, n) be one if pis a 
p-th residue modulo n and zero otherwise, Define a higher 
residue oracle to be a probabilistic polynomial time algo- 
rithm A which takes as input a triple (n,y,p) and returns a 
bit A(n,y,p) such that the following holds: 
There exists a polynomial Q in |n| such Chat, far infinitely 
many values of|n|, one can find a prime pQnj) < B, with; 

Pr{A(n tVt p) = Bp(p,n)> >1-I+I 

where the probability is taken over the random tosses of A 
and its inputs, condftionnaHy to the event that n is (B,a)- 
dense and p is a divisor of #{n) . 

Our Intractability Hypothesis is that there is no higher 
residue oracle. The constant 1 - | comes from the obvious 
stategy for approximating Bp which consists in constantly 
outputting zero- This strategy is successful for a proportion 
1 — J of the inputs. 

33 A security proof 

The security of probabilistic encryption scheme has been 
investigated in [12]. In this paper, the authors introduced 
the notion of semantic security, given two messages mo and 
mi, a message distinguisher is a probabilistic polynomial 
time algorithm Z>, which distinguishes encryptions of mo 
from encryptions of mi. More, accurately, it outputs a bit 
J>(n,<7,0,y) in such a way that, setting 

ft = PriVfa^M) = %€ B(mt)} 

where E(rm) is the set of encryptions of rm, the following 
holds: 

To ere exists a polynomial Q in |n| such that, for ln£mtdy 
many values of JnJ, |6\> — e?i| > £ 

Semantic security is the assertion that there Is no pair of 
polynomial time algorithms F, D such that F produces two 
messages for which D is a message distinguisher. 

Theorem 1 Assume met no higher residttt oracle exists. 
Then, the probabilitic version of the encryption scheme has 
semantic security. 

The proof of this result uses the hybrid technique for which 
we refer to [11], It is tfrhTural in character and we have 
chosen to only indude a sketch it in an appendix to the 
present paper. 

4 Applications and variants 

Even If we do not expect large scale replacement of RSA by 
our scheme, we fed that the latter is worth some academic 
interest Especially, we believe that it opens up new appli- 
cations. We have not yet fully investigated those potential 
applications but we give some suggestions bdow. 



63 



4.1 Traceabffity 

Our proposal could offer some help in the management of 
key escrowing services. Consider the variant of the DifBe- 
HeUman key exchange protocol, where a composite modulus 
n Is used. Such a variant has been studied by various re- 
searchers including Mc Curley in (20], where it is shown that 
some specific choices lead to & scheme that is at least as dif- 
ficult as factoring. Assume further that the modulus n and 
the base far exponentiations g are chosen as described in sec- 
tion L It has been proposed (seee-gil'Q) that g and n could 
be denned by some kind of TTP (Trusted Third Party). 
Now, the user's p ublic key y and his secret key z are related 
byyej* xnodti. It is conceivable to leave the choice of x 
to the user with the provision that x mod a = ID, where 
ID is the identity of the user. This can be checked by the 
TTP upon registration of the key. Thus, we have reached a 
situation where the identity is embedded in the public key 
through a trapdoor, although the actual key is not One 
should not however overestimate the resulting functionality. 
It could be useful in scenarios where traeeabOity is made 
possible via escrowing but where confidentiality cannot be 
broken even with the help of die escrowing services. Al- 
ternatively, it might be used to split traeeabOity and secret 
key recovery between key escrows. Note that the above pro- 
posal requests that 9 is made public: as already observed, 
this does not seem to endanger the scheme. 

4.2 Variants of tha schema 

As is often the case, one can design numerous variants of 
the basic scheme. We will mention two because of their 
potential applications 

Use of moduli with three prime factors As for RSA, it is 
possible to embed three prime factors p, q , r in the modu- 
lus in place of two. The construction is straightforward: the 
small odd prunes p. are split into three groups thus yielding, 
by multiplication, three integers v, 9, w. The three primes 
are then sought among integers of the form to* + 1 (resp. 
26u + 1, resp. 2ao + 1). It seems possible to keep the mini- 
mum size of n to 708 bits, which allows a, b t c to be around 
200 bits. Following an idea of Maurer and Yacobi ([19]). we 
can then hare a complete trapdoor for the discrete logarithm 
with base g: once the a part has been computed, there re- 
mains to compute the logarithm modulo o, 6 and c, which 
is not immediate but well within the reach of current tech* 
nology, since these numbers are 200 bit integers. Again, the 
variant could prove useful in key escrowing scenarios of, say, 
Diffie-HeOman keys, where it might be desirable to have a 
lengthy recovery of the secret key for consumer's protection. 

Multiplicative encryption In this variant, a is made pub- 
lic and encryption applies to messages of length Jfc, tn = 
E^Li™^" 1 * In order to encrypt m t one computes e = 
pT* and apply probabilistic encryption to e. Of course, 
thebandwtth of tab variant is very low: using a 768 bit mod-' 
ulusn and choosing the first 30 odd primes for jxs, we obtain 
a 30 bit input and a 768 bit output. Allowing a larger input 
has drastic conseuuences in terms of the size of n. The value 
of 9 is dose to 2 5 * 0 when the first k primes are used with 
i = 80 but reaches I?** 4 for Jfc » 128 and 2 1 ** for Jk - 160. 
Using the heuristic bound mentioned in section 2.4, we get 
for the length of n something beyond oOOO bits if k is 160. 
This goes down to 2400 bits when 80. 



As a result, the variant just decxibed is not really prac- 
tical and there is little chance that it can ever be adopted 
as an actual encryption scheme. On the other hand, the a- 
phertext c(m) can be used in an encryption scheme a la El 
OamaL The modulus is not prime since it is an RSA mod- 
ulus, but it makes no difference on the user's size. Rom 
a = c(m), he can mana&cture a public key 9 with a corre- 
sponding matching secret key * of bis choice y = h* mod n 
The resulting oyptosystem allows ciphertext traceaJbility in 
the sense of JDesmedt (see [9]). Our proposal enables to 
trace dphertexts by a technique similar to the one used by 
Desmedt, but decreases the size of the modulus from some- 
thing Eke 10030 bits to 2500 bits. The tracing algorithm 
goes as follows; extract from an El Gamal encryption the 
part t* « n r modn and apply the decryption algorithm, 
treating u as a ciphertext. The decryption algorithm wiD 
basically find the original message tn, which provides the 
identity of the user and from which h was built Several 
errors may occur due to the fact that r might have some 
of the pi& as divisors: the corresponding decrypted values 
of nu will be set to 1, regardless' of their original values. 
The correct value can be found if a sample of ciphertexts 
are available or, alternatively, if an error-correction capadty 
has been added to m. Such an error-correction mechanism 
is highly advisable anyway in view of the attacks against 
software key escrow reported in [15]. 

Note that, one can further reduce the size of the expo- 
nent. This is because 40 bits may be considered enough 
for tracing purposes. The value of c goes down to approxi- 
mately 2™ and 1088 bits becomes an acceptable minimum 
length for the modulus. 

5 ChaBenge 

It is a tradition in the cryptographic community to oner 
cash rewards for successful cryptanarysis. More than a sim- 
ple motivation means, such rewards also express the design- 
ers* confidence in their own schemes. As an incentive to 
the analysis of the new scheme, we therefore offer 3 |n| to 
whoever will decrypt : 

c = 13370f «62d81f de3$6dl842f d7eSf claeSb9b449 
bdd00866597e61a£4fb0d939283b04d$bb73f9U 
0d9d61ebO01469OeS67ab89aa8df4a9164cd4c6« 
6df80B06c7c4ced&5cfda97bf7c42cc702S12a49 
ddl96c8746c0e2«f36ca2a««21d4a36aie 

9 = 0b9cf6a7899S9ed4f 36b7Ola506S154zTf4f 1517 
6d731b489787&d26a9e2441SeU14790S0894ba7 
c532adal903c63a84ex7edc29c208a8dU43fbSf7 
d43727b730f20d8ol2cl7cd6cf9ab436ai47cb62 
«9£b8878bfl5204e444ba6ade6132743i9 

n = I459b9617b8a9d*6bd$4341307f 1256oif a241bd 
6Sb9S6dl4078e80dc6116001b83c5f88c7bbcb0b 
db237daac2e76df 5b415d089baa0f d078S 16e60e 
2cdda7c26b8S8777604c&fbdl9f0711bc75ce00a 
5c37e2790b0d9d0ff9625cSab9c75Udi* 

where k = 30 {jh is the i-th odd prime) and the message is 
ASCII-encoded. The challenger should be the first to decrypt 
at least 50% of c and publish the cryptanalysts method but 
the authors are ready to carefully evaluate ad valorem any 
feedback they get. 
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Appendix: Sketch of the Security Proof* 

We show that any message distinguisher can be turned into 
an algorithm that recognizes higher residues. We let £ be a 
distinguisher for two messages mo and mi and start from the 
met that, keeping the above notations, Oq and $\ are signify 
icantly distinct We next use the hybrid technique tor which 
we refer to [11], pp.91-93. Hybrids consist of a sequence of 
random variables Yi, 0 < i < k t such that 

L Extreme hybrids collide with E(mo) and J£(mt) re- 
spectively. 

% Random values of each hybrid can be produced by a 
probabilistic polynomial time algorithm. 



3. There are only polynomial*/ many hybrids. 

In such a situation, [11] shows that D distinguishes two 
neighbouring hybrids. Our hybrids are far' np *l by conrid* 
ering a message jw» such that 

/i< = mo mod pj for j>i and 

pjsmi modpj for / < t 

and letting Yi to be mafbxmly distributed over the set BUn) 
of encryptions of p*. It is easily seen that conditions 1, 2 
and 3 axe satisfied. Thus, for some index t , D significantly 
distinguishes Yt and Yi-\. Set p = p = pi and let //, 
1 < ^* < p, be the unique message such that 

\£ »pmodp/ Cor t^i and =j mod p 

We note that, both m» and rm-i appear among the yfa and 
we show that D cannot distinguish encryptions of any two 
of the pfs. This will yield the desired contramction. 
Let 

*; = Pr{D(n.<r 9 g,y) = l\y€ Efa)} 

and assume that some its significantly exceeds the other 
ones. In other words, «v > sup^ vj + £ for some poly- 
nomial Q and infinitely many values of \n\. We show how 
to predict p-th residuosity: given z, we run V over a targe 
sample N of inputs (n,o*,y) where y = *f 9 with 
x > n and £ < p chosen at random, and we average the 
outputs. Now, if * is a p-th residue, then y simply varies 
over E(pi) t whereas, if z is not a p-th residue, y randomly 
varies over the union of all E{jij)L Thus, in the first case, 
the average ts close to m, whereas, in the second case, it is 

approximately It is easily seen that the difference 

is bounded from below by ^r^- Using the law of large 
numbers, this is enough to make the proper decision on the 
p-th residuosity, with probability as dose to 1 as we wish, 
by using only polyn ami ally large samples. This finishes the 
proof. 
Remarks. 

1. Turning the previous sketch into a complete proof in- 
volves a technical but rather long write-up: especially, a 
precise version of the law of large numbers has to be made 
explicit, e^g. by using the Chebishev inequality. Also, the 

values of ** and ** are not known a priori and should 
be approximated as well using the low of large numbers. We 
urge the interested reader to consult [11] for similar proofs. 

2. The higher residuosity oracle that was built in the proof 
for the sake of contradiction uses inputs <r and g on top of 
n, y and p. Actually, one can check that everything goes 
through, mutatis mutandis, if cr is replaced by 7= iCcs P~ 
Thus ff is not really needed. As for o, as seen in section 2.1, 
ft can be chosen at random: a proper ******* will be spot- 
ted by sampling the corresponding oracle and «*wirhtg its 

correctness. 
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ABSTRACT 

This paper introduces a simple alternative to the hash-and- 
slgn paradigm, from the security point of view but for sign* 
ing short messages, called twinning. A twin signature is 
obtained by signing twice a short message by a signature 
scheme. Analysis of the concept in different settings yields 
the following results: 

• We prove that no generic algorithm can efficiently forge 
a twin DSA signature. Although generic algorithms 
offer a less stringent form of security than computa- 
tional reductions in the standard model, such success- 
ful proofs stilt produce positive evidence in favor of the 
correctness of the new paradigm* 

• We prove in standard model an equivalence between 
the hardness of producing existential forgeries (even 
under adaptively chosen message attacks) of a twin 
version of a signature scheme proposed by Gennaro, 
Halevi and Rabin and the Flexible RSA Problem. 

We consequently regard twinning as an interesting alter- 
native to hash functions for eradicating existential forgery 
in signature schemes. 

Keywords 

Digital Signatures, Provable Security, Discrete Logarithm, 
Generic Model, Flexible RSA Problem, Standard Model. 

1. INTRODUCTION 

The well-known hash and sign paradigm has two distinct 
goals: increasing performance by reducing the size of the 
signed message and Improving security by preventing exis- 
tential forgeries. As a corollary, hashing remains mandatory 
even for short messages. 

From the conceptual standpoint, the use of hash functions 
comes at the cost of extra assumptions such as the conjec- 
ture that for all practical purposes, concrete functions can 
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be identified with ideal black boxes (3] or that under cer- 
tain circumstances (black box groups [15, 21j) a new group 
element must necessarily come from the addition of two al- 
ready known elements. In some settings [11) both models 
- are even used simultaneously. 

This paper investigates a simple substitute to hashing that 
we call twinning. A twin signature is obtained by signing 
twice the same (short) raw message by a probabilistic sig- 
nature scheme, or two probabilistically related messages. 

We believe that this simple paradigm is powerful enough 
to eradicate existential forgery in a variety of contexts. To 
support this claim, we show that no generic algorithm can 
efficiently forge a twin DSA signature and prove that for a 
twin variant of a signature scheme proposed by Gennaro, 
Halevi and Rabin (8] (hereafter GHR) existential forgery, 
even under an adaptively chosen-message attack, is equiva- 
lent to the Flexible RSA Problem (5] in the standard model. 

2. DIGITAL SIGNATURE SCHEMES 

Let us begin with a quick review of definitions and security 
notions for digital signatures. Digital signature schemes are 
the electronic version of handwritten signatures for digital 
documents: a user's signature on a message m is a string 
which depends on m, on public and secret data specific to 
the user and-possibly-on randomly chosen data, in such a 
way that anyone can check the validity of the signature by 
using public data only. The user's public data are called the 
pxibUc key, whereas his secret data are called the secret key. 
The intuitive security notion would be the impossibility to 
forge user's signatures without the knowledge of his secret 
key. In this section, we give a more precise definition of 
signature schemes and of the possible attacks against them 
(most of those definitions are based on [9]). 

2.1 Definitions 

A signature scheme is defined by the three following algo- 
rithms: 

• The hey generation algorithm G, On input 1*, where k 
is the security parameter, the algorithm G produces a 
pair (kp, k,) of matching public and secret keys. Algo- 
rithm G is probabilistic 

• The signing algorithm £. Given a message m and a 
pair of matching public and secret keys (k Pt k,) f £ pro- 
duces a signature a. The signing algorithm might be 
probabilistic. 
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• The verification algorithm V. Given a signature <r, a 
message m and a public key kp, V tests whether a is 
& valid signature of m with respect to kp. In general, 
the verification algorithm need not be probabilistic. 

12 Forgeries and Attacks 

In this subsection, we formalize some security notions 
which capture the main practical situations. On the one 
hand, the goals of the adversary may be various: 

• Disclosing the secret key of the signer. It is the most 
serious attack. This attack is termed total break. 

• Constructing an efficient algorithm which is able to 
sign messages with good probability of success. This 
is called universal forgery. 

• Providing a new message-signature pair. This is called 
existential forgery. 

In many cases this latter forgery, the existential forgery, is 
not dangerous, because the output message is likely to be 
meaningless. Nevertheless, a signature scheme which is not 
existentialry unforgeable (and thus that admits existential 
forgeries) does not guarantee by itself the Identity of the 
signer. For example, it cannot be used to certify randomly 
looking elements, such as keys. Furthermore, it cannot for- 
mally guarantee the non-repudiation property, since anyone 
may be able to produce a message with a valid signature. 

On the other hand, various means can be made available 
to the adversary, helping her Into her forgery. We focus on 
two specific kinds of attacks against signature schemes: the 
no-message attacks and the known-message attacks. In the 
first scenario, the attacker only knows the public key of the 
signer. In the second one, the attacker has access to a list 
of valid message-signature pairs. According to the way this 
list was created, we usually distinguish many subclasses, but 
the strongest is the adoptively chosen-message attack, where 
the attacker can ask the signer to sign any message of her 
choice. She can therefore adapt her queries according to 
previous answers* 

When one designs a signature scheme, one wants to com- 
putationally rule out existential forgeries even under adap- 
tively chosen-message attacks, which is the strongest secu- 
rity level for a signature scheme. 

3* GENERIC ALGORITHMS 

Before we proceed, let us stress that although the generic 
model in which we analyze DSA offers a somehow weaker 
form of security than the reductions that we apply to CHR 
in the standard model, it still provides evidence that twin- 
ning may indeed have a beneficial effect on security. 

Generic algorithms (15, 21], as introduced by Nechaev and 
Shoup, encompass group algorithms that do not exploit any 
special property of the encodings of group elements other 
than the property that each group element is encoded by 
a unique string. Typically, algorithms tike Pollard's p al- 
gorithm [18] fall under the scope of this formalism while 
index-calculus methods do not. 

3. 1 The Framework 

Recall that any Abelian finite group T is isomorphic to a 
product of cyclic groups of the form (Z p *,+), where p is a 
prime. Such groups will be called standard Abelian groups. 



An encoding of a standard group T is an injective map from 
r into a set of bit-strings S. 

We give some examples: consider the multiplicative group 
of invertible elements modulo some prime 9. This group is 
cyclic and isomorphic to the standard additive group T — 
Given a generator g, an encoding <r is obtained by 
computing the binary representation o[x) of g* mod o. The 
same construction applies when one considers a multiplica- 
tive subgroup of prime order r. Similarly, let E be the group 
of points of some non-singular elliptic curve over a finite field 
F, then E is either isomorphic to a (standard) cyclic group 
r or else is isomorphic to a product of two cyclic groups 
Zjj x Zd 3 . In the first case, given a generator G of E t an 
encoding is obtained by computing a(x) = x.G f where x.G 
denotes the scalar multiplication of G by the integer x and 
providing coordinates for o{x). The same construction ap- 
plies when E is replaced by one of its subgroups of prime 
order r. Note that the encoding set appears much larger 
than the group size, but compact encodings using only one 
coordinate and a sign bit ±1 exist and for such encodings, 
the image of a is Included in the binary expansions of inte- 
gers < tr for some small integer t, provided that r is close 
enough to the size of the underlying field F. This is exactly 
what is recommended for cryptographic applications (10). 

A generic algorithm A over a standard Abelian group V 
is a probabilistic algorithm that takes as input an encod- 
ing Ust {ff(xi) f "- *o(zk)} t where each z» is in T. While 
it executes, the algorithm may consult an oracle for further 
encodings. Oracle calls consist of triples {t,jf,c}, where i 
and j are indices of the encoding list and c is ±. The oracle 
returns the string o(xi ± Xj) t according to the value of c 
and this bit-string is appended to the list, unless it was al- 
ready present. In other words, A cannot access an element 
of r directly but only through its name <r(x) and the oracle 
provides names for the sum or difference of two elements 
addressed by their respective names. Note however that A 
may access the list at any time. In many cases, A takes 
as input a pair {er(l),a(x)}. Probabilities related to-such 
algorithms are computed with respect to the internal coin 
tosses of A as well as the random choices of 0 and x. 

The following theorem appears in [21]: 

Theorem 1. Let T be a standard cyclic group of order 
N and let p be the largest prime divisor of N. Let A be a 
generic algorithm over T that makes at most n queries to the 
oracle. If x € T and an encoding a are chosen at random, 
then the probability that A returns x on input {o(\) t a(x)} 
isO(ny P ). 

Proof. We refer to [21] for a proof. However, we wilT 
need, as an ingredient for our own proofs, the probabilistic 
model used by Shoup. We develop the model in the special 
case where N is a prime number r, which is of interest to us. 
Alternatively, we could work in a subgroup of prime order 
r. 

Basically, we would like to identify the probabilistic 6pace 
consisting of o and x with the space 5 n+3 x r, where S is the 
set of bit-string encodings. Given a tuple f*i, < • • ,2n+3,y} 
in this space, z\ and zj are used as <r{l) and <r(x), the suc- 
cessive Z{ are used in sequence to answer the oracle queries 
and the unique value y from V serves as x. However, this 
interpretation may yield inconsistencies as it does not take 
care of possible collisions between oracle queries. To over- 
come the difficulty, Shoup defines, along with the execution 
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of -4, a sequence of linear polynomials Fi(X), with coeffi- 
cients modulo r. Polynomials F\ and Ft are respectively set 
to Fx = t and Fa = and the definition of polynomial Ft 
is related to the /-th query {i, j, e}: F/ = Ft ± F jt where the 
sign ± Is chosen according to c. If Ft is already listed as 
a previous polynomial Fh» then Fr is. marked and X is fed 
with the answer of the oracle at the n-th query. Otherwise, 
zt is returned by the oracle. Once A has come to a stop, the 
value of x is set to 

It is easy to check that the behavior of the algorithm which 
plays with the polynomials F< is exactly similar to the behav- 
ior of the tegular algorithm, if we require that y is not a toot 
of any polynomial Ft —Fj , where i, j range over indices of un- 
marked polynomials. A sequence ,*n+a,y} for which 
this requirement is met is called a safe sequence. Shoup 
showB that, for any ,*n+j}i the set of y such that 
i£n+3tp} ^ not safe has probability 0(n 3 /r). From 
a safe sequence, one can define x as y and a as any encoding 
which satisfies <r(K(y)) = Zi % for all unmarked F«. This cor- 
respondence preserves probabilities. However, it does not 
completely cover the sample space {<r,x} since executions 
such that Ft(x) = Fj(x) t for some indices *, j, such that F< 
and Fj are not identical are omitted. To conclude the proof 
of the above theorem in the special case where N is a prime 
number r, we simply note that the output of a computa- 
tion corresponding to a safe sequence {*i , * • * , v) does 
not depend on y . Hence it is equal to y with only minute 
probability. □ 

3.2 Digital Signatures over Generic Groups 

We now explain how generic algorithms can deal with at- 
tacks against DSA-like signature schemes [6, 20, 16, 10]. 
We do this by defining a generic version of DSA that we 
call GDSA. Parameters for the signature include a stan- 
dard cyclic group of prime order r together with an encod- 
ing a. The signer also uses as a secret key/public key pair 
{x,<r(x)}. Note that we have chosen to describe signature 
generation as a regular rather than generic algorithm, using 
a full description of a. To sign a message m, 1 < m < r the 
algorithm executes the following steps: 

1. Generate a random number u, 1 < u < r. 

2. Compute c «- <r(u) mod r. If c = 0 go to step 1. 

3. Compute d «— u _1 (m + xc) mod r. If d = 0 go to step 
1. 

4. Output the pair {c,'d} as the signature of m. 
The verifier, on the other hand, is generic: 

1. If c $ [1, r- 1| or d $ (l,r- 1), output invalid and stop. 

2. Compute h +- d~ x mod r, h\ *~ hm mod r and hz-+~ 
he mod r. 

3. Obtain v(h\ + fax) from the oracle and compute c' «- 
a[hi + htz) mod r. 

4. If c ^ d output invalid and stop otherwise output valid 
and stop. 

The reader may wonder how the verifier obtains the value 
of a requested at step 3. This is simply achieved by mim- 
icking the usual double- and-add algorithm and asking the 



appropriate queries to the oracle, This yields o(h\) and 
c(kzx). A final call to the oracle completes the task. 

A generic algorithm A can also perform forgery attacks 
against a signature scheme. This Is defined by the ability 
of A to return on input {a(l),a(x)} a triple {m,c, d) € 1^ 
for which the verifier outputs valid. Here we assume that 
both algorithms are performed at a stretch, keeping the same 
encoding list* 

lb deal with adaptive attacks one endows A with another 
oracle, called the signing oracle, lb query this oracle, the 
algorithm provides an element m € T. The signing oracle 
returns a valid signature {c, d) of m* Success of A Is defined 
by its ability to produce a valid triple {m, £,</}, such that 
m has not been queried during the attack* 

Such a forgery can be easily performed against this GDSA 
scheme, even with just a passive attack: the adversary choos- 
es random numbers h\ and Jkg, 1 < h\, /ij < r and computes 
c +- <r{hi + n 2 x) mod r ; Then it defines d = cn^ 1 mod r, 
h = d~ l mod r, and eventually m = dh\ mod r. The triple 
{m»c,d} 6 r 3 is therefore a valid one, unless c = 0, which 
is very unlikely. 

4. THE SECURITY OF TWIN GDSA 
4*1 A Theoretical Result 

The above definitions extend to the case of twin signa- 
tures, by requesting the attacker A to output an m and two 
distinct pairs {c,d} € T a , {c',<f} € T 2 . Success is granted 
as soon as the verifying algorithm outputs valid for both 
triples 1 . We prove the following; 

Theorem 2. Let T be a standard cyclic group of prime 
order r. Let S be a set of bit-string encodings of cardinality 
at least r, included tn the set of binary representations of 
integers < tr, for some t. Let A be a generic algorithm over 
r that makes at most n queries to the oracle. Jfx e T and an 
encoding a are chosen at random, then the probability that 
A returns a message m together with two distinct GDSA 
signatures of m on input (a(l),a(x)} is 0(tn*fr). 

Proof. We cover the non adaptive case and tackle the 
more general case after the proof. We use the probabilistic 
model developed in section 3.1. Let A be a generic attacker 
able to forge some m and two distinct signatures {c,d} and 
cf }. We assume that, once these outputs have been pro- 
duced, A goes on checking both signatures; we estimate the 
probability that both are valid. 

We restrict our attention to behaviors of the full algorithm 
corresponding to safe sequences {21 r • • * , Zn+2, y}- By this, 
we discard a set of executions of probability 0(n 3 /r). We 
let P be the polynomial (md* 1 ) + (cd* 1 )* and Q be the 
polynomial (nwf" 1 ) + (c'<f -1 )A\ 

• We first consider the case where either P or Q does 
not appear in the Fi list before the signatures are pro- 
duced. If this happens for P % then P b included in the 
Fi list at signature verification and the corresponding 
answer of the oracle is a random number z<. Unless 
Zj = c mod r, which is true with probability at most 

1 using[14] the simultaneous square-and-multiply generation 
or verification of two DSA signatures is only 17% slower than 
the generation or verification of a single signature. 
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t/r, the signature is invalid. A similar bound holds for 

• We now assume that both P and Q appear In the F< 
list before A outputs its signatures. We let t denote the 
first index such that Ft = P and J the first index such 
that Pj = Q. Note that both Fi and Fj are unmarked 
(as defined in section 3.1). If i = j, then we obtain 
that md~ l = truf 1 and cdT 1 = cW From this, it 
follows that c = d, d = <f and the signatures are not 
distinct 

• We are left with the case where i / j. We let fljj, 
t < j, be the set of safe sequences producing two sig- 
natures such that the polynomials P, <J, defined as 
above appear for the first time before the algorithm 
outputs the signatures, as Fi and Fj. We consider a 
fixed value tv for {zi t - and let tri be the set 
of safe sequences extending w. We note that Fi and 
f} are defined from w and we write F 4 = a + 6X, 

= a' + 6'X , We claim that n<j n w has proba- 
bility < t/r. To show this, observe that one of the 
signatures that the algorithm outputs is necessarily of 
the form {c,d}, with c — z\ mod r % c~ db mod r and 
m = da mod r. Now, the other signature is {c', <f} and 
since m is already defined we get <£ = ma'" 1 mod r 
and c' = 6'^ mod r< This in turn defines mod r 
within a subset of at most t elements. From this, the 
required bound follows and, from the bound t we infer 
that the probability of Oij is at most t/r. 

Summing up, we have bounded the probability that a safe 
sequence produces an execution of A outputting two valid 
signatures by 0{tn a /r). This finishes the proof. □ 

In the proof, we considered the case of an attacker forg- 
ing a message-signature pair from scratch, A more elab- 
orate scenario corresponds to an attacker who can adap- 
tive^ request twin signatures corresponding to messages of 
his choice. In other words, the attacker interacts with the 
legitimate signer by submitting messages selected by its pro- 
gram. 

We show how to modify the security proof that was just 
given to cover the adaptive case. We assume that each time 
it requests a signature the attacker A immediately verifies 
the received signature. We also assume that the verification 
algorithm is normalized in such a way that, when verifying 
a signature [c,d] of a message m, it asks for a((md~ l ) + 
(cd~ l )x) after a fixed number of queries, say 9. We now 
explain how to simulate signature generation: as before, 
we restrict our attention to behaviors of the algorithm cor- 
responding to safe sequences {n,*- ,Jtn+2»l/}. When the 
(twin) signature of m is requested at a time of the compu- 
tation when the encoding list contains t elements, one picks 
Z{+ 4 and zi+ii and manufactures the two signatures as fol- 
lows: 

1. Let c «— mod r, pick d at random. 

2. Let d «— z<+jfl mod r, pick a" at random. 

3. Output [c,d] and {c\d'} as the first and second sig- 
natures. 



White verifying both signatures, A will receive the ele- 
ments Z{+ q and *i+2f , as 

atfrruT 1 ) + (cd* and c((mi' x ) + (d<T l )x) 

respectively, unless Fi+ 9 or Fi+tg appears earlier in the Ft 
list. Due to the randomness of d and <f, this happens with 
very small probability bounded by n/r. Altogether, the sim- 
ulation is spotted with probability C(n a /r) which does not 
affect the 0(tn 2 /r} bound for the probability of successful 
forgery. 

4J2 Practical Meaning of the Result 

We have shown that, in the setting of generic algorithms, 
existential forgery against twin GDSA has a minute success 
probability. Of course this does not tell anything on the se- 
curity of actual twin DSA- Still, we believe that our proof has 
some practical meaning. The analogy with hash functions 
and the random oracle model [3] is inspiring: researchers 
and practitioners are aware that proofs in the random ora- 
cle model are not proofs but a mean to spot design flaws and 
validate schemes that are supported by such proofs. Still, 
all standard signature schemes that have been proposed use 
specific functions which are not random by definition; our 
proofs seem to indicate that if existential forgery against 
twin DSA is possible, it will require to dig into structural 
properties of the encoding function. This is of some help for 
the design of actual schemes: for example, the twin DSA de- 
scribed in Appendix A allows signature with message recov- 
ery without hashing and without any form of redundancy, 
while keeping some form of provable security. This might be 
considered a more attractive approach than [17] or (1), the 
former being based on redundancy and the latter on random 
oracles. We believe that twin DSA is even more convincing 
in the setting of elliptic curves, where there are no known 
ways of taking any advantage of the encoding function. 

5, AN RSA-BASED TWINNING 

IN THE STANDARD MODEL 

The twin signature scheme described in this section be- 
longs to the (very) short list of efficient schemes provably se- 
cure in the standard model; in the sequel , we show that pro- 
ducing existential forgeries even under an adaptively chosen- 
message attack is equivalent to solving the Flexible EISA 
Problem [5]. 

Security in the standard model implies no ideal assump- 
tions; in other words we directly reduce the Flexible HSA 
Problem to a forgery. As a corollary, we present an efficient 
and provably secure signature scheme that does not require 
any hash function. 

Furthermore, the symmetry provided by twinning is much 
simpler to analyze than Cramer-Shoup's proposal (5] which 
achieves a similar security level, and similar efficiency, with 
a rather intricate proof. 

5.1 Gennaro-Halevi-Rabin Signatures 

In [8] Gennaro, Halevi and Rabin present the following 
signature scheme: Let n be an l-bft RSA modulus [19], H 
a hash-function and y 6 ZJ. The pair. {n,y} is the signer's 
public key, whose secret key is the factorization of n. 

• lb sign ro, the signer hashes e «- H(m) (which is very 
likely to be co-prime with <p(n)) and computes the e-th 
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root of y modulo n using the factorization of m 
a «- y l ' e mod n 

• To verify a given {m,*}, the verifier checks that 
mod u^y. 

Security relies on the Strong RSA Assumption. Indeed, 
if J7 outputs elements that contain at least a new prime 
factor, existential forgery is impossible. Accordingly, Gen- 
naro ei at define a new property that H must satisfy to 
yield secure signatures: division intractability. Division in- 
tractability means that it is computationally impossible to 
rind m, ., . t a* and b such that H(b) divides the product of 
all the £f(oi). In [8], it is conjectured that such functions 
exist and heuristic conversions from collision-resistant into 
division-intractable functions are shown (see also |4]). 

Still t security against adaptively chosen-message attacks 
requires the bash function H to either behave Kike a random 
oracle model or achieve the chameleon property (12). This 
latter property, for a hash function, provides a trapdoor 
which helps to find second preimages, even with some fixed 
part. Indeed, some signatures can be p re-computed, but 
with specific exponents before outputting y: y = x^*' 4 mod 
tt for random primes e< = H(irn t ri). 
- Using the chameleon property, for the t-th query m to 
the signing oracle, the simulator who knows the trapdoor 
can get an r such that fr*(m<,n) = H(m,r) = e<. In the 
random oracle model, one simply defines //{m, r) 4— e,-. 

Then a = x^*"*' = y l ' e< mod n and the signature there- 
fore consists of the triple {m, r, s} satisfying 

s H(m r > = y mod n. 

Cramer and Shoup [5] also proposed a scheme based on 
the Strong RSA Assumption, the first practical signature 
scheme to be secure in the standard model, but with univer- 
sal one-way hash functions; our twin scheme will be similar 
but with a nice symmetry in the description (which helps 
for the security analysts) and no hash-functions, unless one 
wants to sign a long message. 

5.2 Preliminaries 

We build our scheme in two steps. The first scheme resists 
existential forgeries when subjected to no-message attacks. 
Twinning will immune it against adaptively chosen-message 
attacks. 

5.2. 1 Infective Junction into the prime integers. 

Before any description, we will assume the existence of 
a function p with the following properties; given a security 
parameter k (which will be the size of the signed messages), 
p maps any string from {0, \} k into the set of the prime inte- 
gers, p is also designed to be easy to compute and infective, 
A candidate is proposed and analyzed in Appendix B. 

5.2.2 The Flexible RSA Problem and the Strong RSA 
Assumption. 

Let us also recall the Fi&dble RSA Problem [5]. Given an 
RSA modulus n and an element y € Z£, find any exponent 
e > 1, together with an element x such that x* = y mod n. 

The Strong RSA Assumption is the conjecture that this 
problem is intractable for large moduli. This was indepen- 



dently introduced by [2, 7], and then used in many further 
security analyses (e.g. (5, 8]). 

S3 A First GHR Variant 

The first scheme is very similar to GBR without random 
oracles but with function p instead: 

• lb sign m € {0, l} k , the signer computes c p(m) 
and the e-th root of y modulo n using the factorization 
of n 

s 4- y*f* mod n 

• To verify a given {m, 5}, the verifier checks that 

* p(m) mod n = y. 

Since p provides a new prime for each new message (in- 
jectivity), existential forgery contradicts the Strong RSA 
Assumption* However, how can we deal with adaptively 
chosen-message attacks without any control over the output 
of the function p, which is a publicly defined non-random 
oracle and not a trapdoor function either? 

5-4 The Twin Version 

The final scheme is quite simple since it consists in du- 
plicating the previous one: the signer uses two /-bit RSA 
moduli m, n 2 and two elements yi. ya in Z£, and re- 
spectively. Secret keys are the prime factors of the n*. 

• lb sign a message m, the signer probabilistically de- 
rives two messages pi,pj € {0,1}*, (from m and a 
random tape w), computes a *~ pfja) and then the 
e«-th root of yi modulo n,, for t = 1,2, using the fac- 
torization of the moduli: 

{s\ *— y\ /Cl mod ni, S3 *— y*** m °d n *} 

• To verify a given {m, w, * 1 , a 3 } » the verifier computes 

pi and (i2> then checks that mod m = y t , for 
i=l,2. 

To prevent forgeries, a new message must involve a new 
exponent, either ei or ea, which never occurred in the sig- 
natures provided by the signing oracle. Therefore, a first 
requirement is that pi and pa define at most one message 
m, but only if they have been correctly constructed. Thus, 
some redundancy is further more required. 

We thus suggest the following derivation, to get pi and 
pa from m € {0.1}*' 2 (we assume k to be even): one. 
chooses two random elements a, 6 € {0, l}^ 3 , then pi = 
(m $ a)\\(m © 6) and pa = a\\b. 

Clearly, given pi and p 3 , one gets back M = pi Q p 2 , 
which provides a valid message if and only if the redundancy 
holds: A? = jVI, where S and S denote the two fc/2-bit 
halves of a fc-bit string 5, the most significant and the least 
significant parts respectively 

5.5 Existential Forgeries 

Let us show that existential forgery of the twin scheme, 
with above derivation process, leads to a new solution of the 
Flexible RSA Problem: 

LEMMA 1. After q queries to the signing oracle, the prob- 
ability that there exist a new message m and value* a, b, 
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which lead to ui = (m ffi a)||(m © 6) and Hi = a||fr, «ttcA 
Mat 60 (A ei = and e* = p(/xa) already occurred in the. 
signatures provided by the signing oracle is less thanq* /2 k l* . 

Proof. Let {m^o^A*,^,^} denote the answers of 
the signing oracle. Using the injectiyity of p, the existence 
of such m, a and 6 means that there exist indices i and j for 
which 

(m©a)||(me6)=^i = p M = (m* ©a<)ll(m< ©6<) 
a||6 = /i 3 = P2j = aA\bj- 

Then 

a ©6 = (m©a)©(m©6) — (to< ©a<)© (m ( ©6 4 ) = a< ©6<, 
and 

o©6 = ©ty. 

Therefore, for a j X (the case < > / Is similar), the new 
random elements a.) , 6, must satisfy aj ©fy = <n ©6*. Since 
it is randomly chosen by the signer, the probability that this 
occurs for some t < j is less than (j - l)/2 fc/a . 

Altogether, the probability that for some j there exists 
some i < j which satisfies the above equality is less that 
q*/2 x 2~*'*. By symmetry, we obtain the same result if we 
exchange i and j. 

The probability that both exponents already appeared is 
consequently smaller than q 7 /2* /2 . □ 

To prevent adaptively chosen-message attacks, we need 
no trapdoor property for p, nor random oracle assumption 
either. We simply give the factorization of one modulus to 
the simulator, which can use any pre-computed exponenti- 
ation with any new message, as when chameleon functions 
are used (8]. 

5.6 Adaptively Chosen-Message Attacks 

Indeed, to prevent adaptively chosen-message attacks, one 
just needs to describe a simulator; our simulator works as 
follows: 

• The simulator is first given the moduli n^nj and the 
elements y 4 € Zt^, € Z^ 3 , as well as the factor- 
ization of n,, where 7 is randomly chosen in {1,2}. 
To simplify notations we assume that 7 = 1. And 
the following works without loss of generality since the 
derivation of ui and u% is perfectly symmetric: they 
are randomly distributed, but satisfy /ii © ^3 = m\\m 
(it is a perfect secret sharing). 

• The simulator randomly generates q values e 2j - «— 
PiPw)* with randomly chosen /i 34 {0, l} fc for jf = 
1, . . . , q and computes 

The new public key for the signature scheme is the 
following: the moduli rn,n* with the elements y u z in 
ZJij and ZJ a respectively. 

• For the j-th signed message m, the simulator first gets 
(a||6) «- (m||m) © inj. It therefore computes u\ «- 
a||6, and thus jt* = (m©a)||(m©6). 

Then, it knows s 2 = ^w* 2 *' mo d n 2l and computes 
Si using the factorization of ni. 



Such a simulator can simulate up to q signatures, which 
leads to the following theorem. 

THEOREM 3. Let us consider an adversary against the 
twin~GHR scheme who succeeds in producing an existen- 
tial forgery, with probability greater toon e, after q adaptive 
queries to the signing oracle in time t, then the flexible RSA 
Problem can be solved with probability greater than e* within 
a time bound t\ where 

e ' = l( e "2^) « rf '«'-* + 0 <« x ***>- 

Proof. Note that the above bounds are almost optimal 
since e J « c/2 and t' ~ 2*. Indeed, the time needed to 
produce an existential forgery after q signature queries is 
already In C(q x (|ni| a + |n 2 p)Jfe). To evaluate the success 
probability, q is less than say 2 , but k may be taken greater 
than 160 bits (and even "much more). 

To conclude the proof, one just needs to address the ran- 
dom choice of 7. As we have seen in Lemma 1, with proba- 
bility greater than e - gV 2 * /2 t one of the exponents in the 
forgery never appeared before. Since 7 is randomly chosen 
and the view of the simulation is perfectly independent of 
this choice, with probability of one half, e = is new. Let 
us follow our assumption that 7=1, then 

a e = 53 = z = yj mod n% % 

where * = J\ JmX _ 4 eaj. Since e is new, it is relatively 
prime with n, and therefore, there exist u and v such that 
ue + tm = 1: let us define z ~ s v mod na, 

- Ms")' = y\-"*s" = y*(yS)~ v (s e r = W mod n 2 . 

We thus obtain an e-th root of the given j& modulo nj, for 
a new prime e. □ 

5.7 More Signatures 

One may remark that the length of the messages we can 
sign with above construction is limited to fc/2 bits, because 
of the required redundancy. But one can increase the size, by 
signing three derived messages: in order to sign m € {0, 1} , 
one chooses two random elements a, b e {0, 1}* /3 (we still 
assume k to be even), and signs with different moduli 

♦ui = m©(a||6) 

*i2 « a\\b 

u 3 = m©(6||a). 

6. CONCLUSION 

AND FURTHER RESEARCH 

We proposed an alternative to the well-known hash-and- 
sign paradigm, based on the simple idea of signing twice 
(or more) identical or related short messages. We believe 
that our first investigations show that this is a promising 
strategy, deserving further study. 

A number of interesting questions remain open. First, 
from the efficiency point of view, which is a frequent concern, 
we are aware that the current proposals do not deal with 
either the computational cost, or the communication load, in 
an efficient way. Thus, for example, can the number of fields 
in a twin DSA be reduced from four ({c,d} and {c?,d*}) to 
three or less? Can we also suppress some fields in the twin- 
GHR, or sign Jb-bit long messages with only two signatures? 
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Finally, can an increase in the number of signatures (e.g. 
three instead of two) yield better security bounds? 
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APPENDIX 

A. TWIN SIGNATURES 

WITH MESSAGE RECOVERY 

In this appendix, we describe a twin version of the Nyberg- 
Rueppel scheme (17] which provides message recovery. Keep- 
ing the notations of section 4.1: 

1. Generate a random number u, I < u < r. 

2. Compute c «- cr{u) + m mod r. If c = 0 go to step 1. 

3. Compute an integer d+—u — cx mod r. 

4. Output the pair (c,d) as the signature. 

In the above, /• is what is called in (10] a message with 
appendix. It simply means that it has an adequate redun- 
dancy; The corresponding verification is performed by the 
following (generic) steps: 

1. If c i [1, r - 1] or d g [Q,r - 1), output invalid and stop. 

2. Obtain a(d + ex) from the oracle and compute 7 «— 
o(d + car) mod r . 

3. Check the redundancy oFm «— c - 7 mod r. If in- 
correct output invalid and stop; otherwise output the 
reconstructed message m, output valid and stop. 

In the twin setting, signature generation is alike but is per- 
formed twice, so as to output two distinct signatures. How- 
ever, no redundancy is needed. The verifier simply checks 
that the signatures are distinct and outputs two successive 
versions of the message, say m and m'. It returns valid 
if m = m* and invalid otherwise. The security proof is 
sketched here, we leave the discussion of adaptive attacks 
to the reader. 

We keep the notations and assumptions of section 4 and 
let A be a generic attacker over T which outputs, on input 
{<r(l),<r(a:)}, two signature pairs {c,d} t {J, a*} and runs the 
verifying algorithm that produces from these signatures two 
messages m, m' and checks whether they are equal. We 
wish to show that, if x € T and an encoding 0 are chosen at 
random, then the probability that m = m' is 0(fcn a /r). 

As before, we restrict our attention to behaviors of the full 
algorithm corresponding to safe sequences {zi,*-* ,2«,y}. 
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We let P, Q be the polynomials d + cX and df + c*X . We 
first consider the case where either P or Q does not appear 
in the Ft list before the signatures are produced. If this 
happens for P, then, P Is Included In the F< list at signature 
verification and the corresponding answer of the oracle is ft 
random , number Zi. Since m Is computed as c - h mod r, 
the probability that m = ro' is bounded by t/r. A similar 
bound holds for Q. 

We now assume that both P and Q appear in the Ft 
list before A outputs its signatures. We let t denote the 
first index such that F« = P and j the first index such 
that Fj = Q. Note that both Fi and Fj are unmarked (as 
defined in section 3.1). If i = i, then we obtain that c= c* 
and d = df. From this, it follows that the signatures are not 
distinct. 

As in section 4, we are left with the case where i ^ j 
and we define ft^-, i < j, to be the set of safe sequences 
producing two signatures such that the polynomials P, Q, 
defined as above appear for the first time before the al- 
gorithm outputs the signatures, as Fi and Fj. We show 
that, for any fixed value ut = {z\ % • * • , H{j D w has 

probability < t/r, where t£r is defined as above. Since we 
have m — c — z% mod r and m' = C* — zj mod r, we obtain 

= </ — c + Zi mod r, from which the upper bound follows. 
Horn this bound, we obtain that the probability of Clij is at 
most t/r and. taking the union of the various 0<js, we con- 
clude that the probability to obtain a valid twin signature 
is at most 0(tn 3 /r). 

B. THE CHOICE OF FUNCTION P 
B.l A Candidate 

The following is a natural candidate: 
p:{0,l} fc - V 

m i-* nextprlae(m x 2 r ) 

where t is suitably chosen to guarantee the existence of a 
prime in any set |ro x 2 T , (ro + 1) x 2 T (, for ro < 2*. 

Note that the deterministic property of nextprime is not 
mandatory, one just needs it to be injective. But then, the 
preimage must be easily recoverable from the prime: the 
exponent is seat as the signature, from which one checks 
the j>rimality and extracts the message (message-recovery). 

B.2 Analysis 

It is clear that any generator of random primes, using m as 
a seed, can be considered as a candidate for p. The function 
proposed above is derived from a technique for accelerating 
prime generation called incremental search (e.g. (13], page 
148). 

1. Input: an odd *-bit number no (derived from ro) * 

2. Test the s numbers no, no + 2, . . . , no + 2(* - 1) for 
primality 

Under reasonable number-theoretic assumptions, if s = 
c- In 2*, the probability of failure of this technique is smaller 
than 2e~ 2c , for large k. 

Using our notations, in such a way that there exists at 
least a prime in any set [m x 2 T , (ro + 1) x 2 T |, but with 
probability smaller than 2~*°, we obtain from above formu- 
lae that c ^ 40, and 2 T >40ln2* +T+l . Therefore, a suitable 



candidate is r s eiogj Jfe, and less than 20* primality tests 
have to be performed. 

B3 Extensions 

B. J. I Collision-resistance: 

To sign large messages (at the cost of extra assumptions), 
one can of course use any collision-resistant hash-function h 
before signing (using the classical hash-and-eign technique)* 
Clearly, the new function in p{h{m)) is not mathemati- 
cally injective, but just computationally injective (which is 
equivalent to collision-resistance) , which is enough for the 
proof. 

B.3.2 Division intractability: 

If one wants to improve efficiency, using the division- 
intractability conjecture proposed in [8], any function that 
outputs fc-bit strings can be used instead of p. More pre- 
cisely: 

Definition (Division Intractability). A function H is 
said (n, i/, T)~divi$ion intractable if any adversary which runs 
in time t cannot find, with probability greater than u t a set 
of elements ai, . . , , a n and b such that H(b) divides the 
product of all the H(oh). 

As above, that function p would not be injective, but just 
collision-resistant, which is enough to prove the following: 

Theorem 4. Let us consider the twin-CHR scheme where 
p is any (q t e, t)-division~intractable hash function. Let us 
assume that an adversary A succeeds in producing an ex- 
istential forgery under an adoptively chosen-message attack 
within time t. and with probability greater than e, after q 
queries to the signing oracle. Then one can either contra- 
dict the dtvision-tntmctabUxty assumption or solve the Flex- 
ible RSA Problem 'with probability greater than e' xmthin a 
time bound t' t where 




and t'=t + Q(qx{? x*). 
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Abstract: The signature generation phase of most 
DLF-based signature s chemes (for t^t^rf"* 
SduunitO], H«Gainal{41 or the newly 
standardized D&AJ3D includes the time- 
consuming computation of r= g* modp where * 
Is random. 

This paper introduces a new computational 
strategy that can apply in this particular context ; 

A botch exponentiation technique which allows 
the generation of large sets of exponentials 
without introducing any bias between the is (that 
is, the signer can batch-compute the exponentials 
corresponding to arbitrarily imposed powers -for 
instance by an external random number 
generator). Our method offers real improvements 
over the prior art with various time and memory 
trade-offs. 



L Introduction 

la many DIP-based signature schemes 1 the signer performs the 
operation r» g K modp where k is random As the signer is often 
the u weak party " in the signature protocol, several authors tried 
to accelerate the exponentiation by pre-cornputing values (11, (51 
or sutxontracting a part of the exponentiation workload to the 
verifier £6] (provided that a set of precautions Is taken into 
consideration). Except the fact that some of these algorithms were 
broken [7], [8], extra memory storage is frequently an unrealistic 
assumption. 

In this paper, we investigate a strategy for improving the 
generation of r : the method (providing Improvements ranging 
from 42% to 85% over the square-&-multipfy algorithm) can 
apply to the batch generation of rued-g-based signatures without 
introducing any bias into the exponents (that is, we assume that 
the As are Imposed to the signer by some random source). We 
assume that no pre-cornputation is allowed other than what 
needed to execute similar size basic square-it multiply. The new 
method may as well open the way to interesting developments for 
accelerating the computation of discrete logarithms. 

JVnruuionto crake digtUl/hud cosies oftUorpict of this mticrisl for 

persoatt or cUssroom um a greeted wiihout &c provided that the copies 

ere not nude or aittiftnaed for profit or c o m mer ci al aovsauge, (be copy- 

r^ht notice, the title of the pWx*ikm »t4 hm d*u tppeir, w&tU 

given thti copyright U by j>emus«ioo of (he ACM, Int. To eopy ctnerwise, 

to lepvbUifc, lo post on servers or to redisutbute to Uiu, rtquires^eeiCc 

penmsson end/or tee* 

CCS *96, New Delhi, India 

• 1996 ACM <W9791-82*O/96/03..$3.5O 



2. Unbiased Batcb-ExponeotiatiQii 

The simultaneous signature of many messages by a shop terminal 
or the processing of dectnmic documents by an rnlmfnfetration 
frequently involves massive computations consisting in die 
itpetitiflo of small operations. The recombination of these 
computations tor nurumizing the computational effort is thus an 
interesting research direction (see for instance [10]). 
The batch exponentiation technique proposed hereafter is built 
around the following observation : since the exponents are 
random, the uniform distribution of ones in different exponents is 
expected to have some matching patterns. Considering this fact, 
our batcrj-exponentiation strategy consists in rmni miring the 
rioWs woAtoad hv CTxmentiating the Intersection separately 
mi reftffre ft? coTjy^rjdJrgJMa in the initial exponents. 

U Parallel tquare*&*muttipty (straightforward} 

Let n be the size of the exponents and N the number of 
exponentials to be computed. The usual method to generate the rs 
consists in calculating successive squares of g and performing the 
required multiplications selected by the bits of each *,* (about nfl 
rrmliiptications). 

Let S« {fy |i«V-.fV} be a set of random powers. The 
corresponding set of exponentials 

can be computed by performing the successive squares of g only 
once. Thus, the total computational effort mainly relies oa the 
average number of multiplications, depending on the Hamming 
weight of each random exponent 

The algorithm PSM(S,rt). of complexity 

£(tf)=Vv*(n/2-l)+(/i-i) 

using N + 1 registers, is : 

for i«-l to* 
forj4-0ton-l 

fori*-] to N if 1 then /j*- (^gjmodp 
g4-g*gmodp 

n-1 

where */= I */f/P'. 
12 The Basic Strategy 

Denoting S a set of >V random powers k, the strategy to generate 
the related exponentials is the following : 

OCvtStotoT~lN/L]se*sh=[kj}^ L where i-£S a 
2^-1 

6UtrX*fc) = U*M wfcs»*n,i are sjt's subsets with jj^f *0 
M 



1 for instance [3J. [4] and [tOJ 

2 L value depends on the exponent length; further explanation will 
be found in 3.2 
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OF6rn«-l tof 

0>For<4-lto2*--I 



OF6rC4-L-lto 1 

Fori «-l to 2^-1 

ifGi/d(j^)-Cthen 

OKs*^}*) 

0Fori«-ltoL 

^'"Il^/y where rye* 

The idea is to operate on each subset, reseting die common bits of 
the powers, and compute the exponentials together to save both 
squarrings and multiplications. The following section will 
describe the method for a reduced only two-power set 

23 Exponent Combination Method 

Hereafter, we only consider the number of multiplications 
required to compute the rj's assuming that it is possible to 
c alculat e the squares only once by the previously described 
parallel method. 

Denoting 

i i 

and assuming that g a mod p and g b mod p are to be computed, 
tetc-Xa#2'\ 
i 

If a and b are randomly chosen, one should expect that: 

i 2 i 4 

Given the fact that: 

w(a - c) + w{b - c) + h<c ) i w(a ) + h<*) 

(where v denotes the Hamming weight), our strategy consists in 
computing : 

G 8 *g*®c ro0 dp 
Gb » g b ® c mod p 
G c *= g c mod p 

to obtain 

{ r a ~ G oPc moc * p = 8 a P 
rj, = G^G C mod p = g* mod p 

The gain for a set of N signatures is therefore statistically N(n/4 - 
1) multiplications, which tends to 25% of the total multiplications 
required to generate a set of signatures with the parallel square- 
&-multipfy, if we simply apply this strategy to all signatures 
grouped by pairs as illustrated in the following table : 





PSM 


Exponent Combination 




5 ft/2-1 


sa/2-/i/4-1 




5/1/2-1 


sn/2-n/4-l 




none 


sn/4-1 


Total 


3n-2 


= 3«/4-3 



Table 1 : PSM and batch exponcntfattan performance 

The computational effort required to generate the set 
G={g*i owd/>|/=U t /v} 

is about /V/2<3n/4-3))+ii/* but implies to use 3A//2 sheQ>y 
bit registers which may not be practical in some situations. In the 
following we win present an optimization of our batch strategy 
and achieve comparison with BrickeU, Gordon, McCurtey and - 
Wilson algorithm in M, exhibiting when our strategy is more 
convenient and suitable. 



3. Improvement and Perfofmapw 
3J IrCCW algorithm 

The precomputation technique proposed by BrickeU and al. in [t] 
is based on the following observation : in [1 1] it was proposed to 
precompute the set of g^'to reduce the computational effort 
increasing the storage amount There is no reason to consider 
powers of 2, 

The idea is to find a decomposition 
m-1 

i=0 

where 0£aj £h for 0 £ i £ m , then we can compute 

<m a 

where c^eria,^^ • 

The algorithm directly derived from this achieves the computation 
of g* with only m+n-2 multiplications, but required that the 
values of % x i mod p have been previously stored To give an 
overview of the performances of this algorithm, one can remark 
that to generate a g k with a 160-bit Jk, say for a Schnorr or DSS 
signature, using a base 16 representation for the numbers, the 
BGCW algorithm produces g* at the cost of only 50 
multiplications but requires also the storage of 40 n-bit numbers. 
Considering a base 32 notation, one can save a tew storage (about 
2 Kbytes rather than US previously) but the number of 
multiplications grows to 60. An improvement based on the notion 
of a basic digit set improves drastically the speed performance 
since the number of multiplications falls to about 36 but implies 
the storage of more than 200 numbers. The main drawback of the 
method is clearly the rrunimum storage capacity needed to achieve 
an exponentiation. 

32 Optimizing the exponent cornbination 

The total number of computations required to produce the set of 
g*s will depend mainly on the multiplications to be calculated 
since the squarings are done once for alt 
Considering that we want to group a n-bit exponents from a large 
set of lvalues together rather than only joining them by pairs, the 
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mam issue is to find the best combination strategy to reduce the 

multiplications to be done. 

The number of multip£cstions per g* U in average 

Af(«^)=5^Lzl)^2a-l-1 

and the squaring effort can be divided between aH the fa. 

The analysis of the function representation (see Annex 1) for 
usual exponent lengths (160-bit and 512-btO give integer 
solutions which minimize this quantity. The number of registers 
required to achieve the computation increasing with the number of 
elements in the set (since to compute squaring* once we must 
calculate all (he g*s together), the best values appear to be 4 tor 
160-bit values and 5 for 512-bit values. The final choice for a 
riwticatftd compulation relies mainly on the memory capacity of 
the machine which generates the g* s, 

33 Performance Overview 

The various strategies achieving several time and memory trade- 
offs are to be considered. The Tables 1 and 2 give the different 
trade-offs for 160-bit (Schnorr, DSS) and 512-bit (E3- Gamal 
BrickcU-McCuriey) exponents, considering that p is a 512-bit 
prime modulus. 



Set 
She 


Subset 
Site 


Storage (ByUi) 


multiplications per 
computation 


2 


2 


316 


141 


3 


3 


652 


103 


4 


2 


568 


101 


4 


4 


U24 


84.5 


12 


3 


2.416 


63 


12 


4 


3.844 


57.83 


36 


3 


7.120 


54.11 


36 


4 


11.404 


48.94 


108 


4 


34.084 


45.98 



Table 2 : performances with 160-bU exponent 



Se* 
Size 


Subset 
Site 


Storage (Bytes) 


jtMuUsplkations per 
computation 


2 


2 


448 


449 


3 


3 


960 


323 


4 


4 


1.984 


255 


5 


5 


4.032 


216.6 


60 


3 


17.984 


160.87 


60 


4 


28364 


135.53 


60 


S 


47.680 


12X73 


200 


5 


158.784 


116.76 



Table 3 i performances with 512-bit exponent 



Compare to the Squares-Multiply and BCCW algorithms, the 
Batch Exponentiation strategy provides nice improvements to 
save computation and memory. The pairs grouping provides a 42 
% gain on the total computation at the cost of only 2 n-bit 
registers, while the BGCW implies at least a 2 Kbytes storage. U* 
technique is perfectly suited to low-memory environment where 
memory cost is high; furthermore, the exponent re-combination is 
easy to perform on any machine. On the other hand, BGCW 
algorithm is very efficient when at least a few Kbytes of 
perrnanent memory are available. 



An adaptation of Batch Exponentiation tailored for high-speed 
transmission (see Annex 2) even provides greater improvements 
by sub-contracting the squaring effort to an external device 
assuming nothing on his security. . 



4. Cottdsslon, Extensions and Open Questions 

We presented a strategy which can accelerate the generation of 
DLP-bascd dgmuurcs. The main characteristics of the batch- 
exponentiation technique presented in this article are surnmarized 
in the following table. 



scheme** 
efforts 


Batch 

(memory) 


Batch 
(time) 


Schnorr 


141 N 


45.98 N 


BLGamal 


449 AY 


116.76 N 



Table 4 : exponentiation pevf onnance 



Several open questions appear interesting to explore to further 
improve the proposed strategies : 

• For a power <€t^ f try to find a such that A' -a ^ + it where 
the hamming weight of k* is sigmficalrvely small. Since 
computations are done modulo q t this transformation of k 
does not have any impact on the result itself but may well 
reduce the computation workload. 

• Find an algorithm such that the construction of the subsets is 
optimal, that is the ordering of the fa results in as few 
computations as possible. 
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Figure 1 : 160-bit exponent - Best integer solution « 4 
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Figure 2 : 512-bit exponent - Best Integer solution = 5 



Appendix 2 : Sub-contracting squaring* with high-speed 
transmission 



Using a high-speed transmission interface such that of a PCMCIA 
card, one can subcontract the square computation and rely on the 
device with the game level of security. 

The strategy remains the same, grouping the exponents and 
computing the g*'s, except that the squaring are computed by a 
genuine device, assuming nothing on his tamper-resistance. The 
device that shall compute g* values has a certificate Con the set 
of |g 2 ' mod/? 1 1 £ j£&(p)j such as en iterative hashing of the 
whole set 

Denoting Senderiht computing device in charge of the squaring 
effort and Receiver the exponentiation machine, the protocol is 
the following : 

Sender Receiver 



For /=0 to «-! 

Send j e* Receive s 

(use if needed) 

jepmodp 

l=SHA(fy) 

If C«/ accept 
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Abstract Smartcards are the most secure portable computing 
device today. They have been used successfully in applications 
involving money, and proprietary and personal data (such as 
banking, healthcare, insurance, etc,). As smartcards get more 
powerful (with 32-bit CPU and more than I MB of stable 
memory in the next versions) and become multi-application, 
die need for database management arises. However, smart- 
cards have severe hardware limitations (very slow write, very 
little RAM, constrained stable memory, no autonomy, etc.) 
which make traditional database technology irrelevant. The 
major problem is scaling down database techniques so they 
perform well under these limitations. In this paper, we give an 
in-depth analysis of this problem and propose a PicoDBMS 
solution based on highly compact data structures, query ex- 
ecution without RAM, and specific techniques for atomicity 
and durability. We show the- effectiveness of our techniques 
through performance evaluation. 

Key words: Smartcard applications - PicoDBMS - Storage 
model - Execution model - Query optimization - Atomicity 
- Durability 



1 Introduction 

Smartcards are the most secure portable computing device to- 
day. The first smartcard was developed by Bull for the French 
banking system in the 1980s to significantly reduce the losses 
associated with magnetic stripe credit card fraud. Since then, 
smartcards have been used successfully around the world in 
various applications involving money, proprietary data, and 
personal data (such as banking, pay-TV or GSM subscriber 
identification, loyalty, healthcare, insurance, etc.). While to- 
day's smartcards handle a single issuer-dependent application, 
the trend is toward multi-application smartcards 1 . Standards 
for multi-application support, like the JavaCard [36] and Mi- 
crosoft's SmartCard for Windows [26], ensure that the card 
be universally accepted and be able to interact with several 



service providers. This should make smartcards one of the 
world's highest-volume markets for semiconductors [14]. 

As smartcards become more and more versatile, multi- 
applicauon, and powerful (32-bit processor, more than 1 MB 
of stable storage), the need for database techniques arises. Let 
us consider a health card storing a complete medical folder 
including the holder's doctors, blood type, allergies, prescrip- 
tions, etc. The volume of data can be important and the queries 
fairly complex (select, join, aggregate). Sophisticated access 
rights management using views and aggregate functions are re- 
quired to preserve the holder's data privacy. Transaction atom- 
icity and durability are also needed to enforce data consistency. 
More generally, database management helps to separate data 
management code from application code, thereby simplifying 
and making application code smaller: Finally, new applica- 
tions can be envisioned, like computing statistics on a large 
number of cards, in an asynchronous and distributed way. Sup- 
porting database management on the card itself rather than on 
an external device is the only way to achieve very high secu- 
rity, high availability (anywhere, anytime, on any terminal), 
and acceptable performance. 

However, smartcards have severe hardware limitations 
which stem from the obvious constraints of small size (to 
fit on a flexible plastic card and to increase hardware se- 
curity) and low cost (to be sold in large volumes). Today's 
microcontrollers contain a CPU, memory - including about 
96 kB of ROM, 4 kB of RAM, and up to 128 kB of stable stor- 
age like EEPROM - and security modules [39]. EEPROM 
is used to store persistent information; it has very fast read 
time (60-100 ns) comparable to old-fashion RAM but very 
slow write time (more than 1 ms/word). Following Moore's 
law for processor and memory capacities, smartcards will get 
rapidly more powerful. Existing prototypes, like Gemplus's 
Pinocchto card [16], bypass the current memory bottleneck 
by connecting an additional chip of 2 MB of Flash memory to 
the microcontroller. Although a significant improvement over 
today's cards, this is still very restricted compared to other 
portable, less secure, devices such as Personal Digital Assis- 
tants (PDA). Furthermore, smartcards are not autonomous, 
i.e., have no independent power supply, thereby precluding 
asynchronous and disconnected processing. 



Everyone would probably enjoy carrying far fewer cards. 
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These limitations (tmy RAM, little stable storage, very 
costly write, and lack of autonomy) make traditional database 
techniques irrelevant Typically, traditional DBMS exploit sig- 
nificant amounts of RAM and use caching and asynchronous 
I/Os to reduce disk access overhead as much as possible. With 
the extreme constraints of the smartcard, the major problem is 
scaling down database techniques. While there has been much 
excellent work on scaling up to deal with very large databases, 
e.g., using parallelism, scaling down has not received much at- 
tention by the database research community. However, scaling 
down in general is becoming very important for commodity 
computing and is quite difficult [18]. 

Some DBMS designs have addressed the problem of seal* 
ing down. Light versions of popular DBMS like Sybase Adap- 
tive Server Anywhere [37], Oracle 8i Lite [30] or DB2 Every- 
where [20] have been primarily designed for portable comput- 
ers and PDA. They have a small footprint which they obtain 
by simplifying and componentizing the DBMS code. How- 
ever, they use relatively high RAM and stable memory and do 
not address the more severe limitations of smartcards. ISOL's 
SQUava Machine DBMS [13] is the first attempt towards a 
smartcard DBMS while SCQL [24], the standard for smartcard 
database language, emerges. While both designs are limited to 
single select, they exemplify the strong interest for dedicated 
smartcard DBMS. 

In this paper, we address the problem of scaling down 
database techniques and propose the design of what we call a 
PicoDBMS, This work is done in the context of a new project 
with Bull Smart Cards and Terminals. The design has been 
made with smartcard applications in mind, but its scope ex- 
tends as well to any ulna-light computer device based on a 
secured monolithic chip. This paper makes the following con- 
tributions: 

• We analyze the requirements for a PicoDBMS based on a 
typical healthcare application and justify its minimal func- 
tionality. 

• We give an in-depth analysis of the problem by considering 
the smartcard hardware trends and derive design principles 
for a PicoDBMS. 

• We propose a new pointer-based storage model that inte- 
grates data and indices in a unique compact data structure. 

• We propose query execution techniques which handle 
complex query plans (including joins and aggregates) with 
no RAM consumption. 

• We propose transaction techniques for atomicity and dura- 
bility that reduce the logging cost to its lowest bound and 
enable a smartcard to participate in distributed transac- 
tions. 

• We show the effectiveness of each technique through per- 
formance evaluation. 

This paper is an extended version of [7]. In particular, the 
section on transaction management is new. The paper is orga- 
nized as follows. Section 2 illustrates the use of take-away 
databases in various classes of smartcard applications and 
presents in more detail the requirements of the health card 
application. Section 3 analyzes the smartcard hardware con- 
straints and gives the problem definition. Sections 4-6 present 
and assess the PicoDBMS* storage model, query execution 
model, and transaction model, respectively. Section 7 con- 
cludes. 



2 Smartcard applications 

In mis section, we discuss the major classes of emerging smart- 
card applications and their database requirements. Then, we 
illustrate these requirements in further detail with the health 
card application, which we will use as reference example in 
the rest of the paper. 

2 A Database management requirements 

Table 1 summarizes the database management requirements 
of the following typical classes of smartcard applications: 

• Money and identification: examples of such -applications 
are credit cards, e-purse, SIM for GSM, phone cards, trans- 
portation cards. They are representative of today's applica- 
tions, with very few data (typically the holder's identifier 
and some status information). Querying is not a concern 

' and access rights are irrelevant since cards are protected by 
PIN -codes. Their unique database management require- 
ment is update atomicity. 

• Downloadable databases: these are predefined packages 
of confidential data (e.g., diplomatic, military or business 
information) that can be downloaded on the card - for ex- 
ample, before traveling - and be accessed from any termi- 
nal. Data availability and security are the major concerns 
here. The volume of data can be important and the queries 
complex. The data are typically read-only. 

• User environment: the objective is to store in a smartcard 
an extended profile of the card's holder including, among 
others, data regarding the computing environment (PC's 
configuration, passwords, cookies, bookmarks, software 
licenses, etc.), an address book as well as an agenda. The 
user environment can thus be dynamically recovered from 
the profile on any terminal. Queries remain simple, as data 
are not related. However, some of the data are highly pri- 
vate and must be protected by sophisticated access rights 
(e.g., the card's holder may want to share a subset ofher/his 
address book or bookmark list with a subset of persons). 
Transaction atomicity and durability are also required. 

• Personal folders: personal folders may be of a different na- 
ture: scholastic, healthcare, car maintenance history, loy- 
alty. They roughly share the same requirements, which 
we illustrate next with the healthcare example. Note that 
queries involving data issued from different folders can 
make sense. For instance, one may be interested in discov- 
ering associations between some disease and the scholastic 
level of the card holder. This raises the interesting issue of 
maintaining statistics on a population of cards or mining 
their content asynchronously. 



2 J The health card application 

The health card is very representative of personal folder appli- 
cations and has strong database requirements. Several coun- 
tries (France, Germany, USA, Russia, Korea, etc.) are devel- 
oping healthcare applications on smartcards [1 1], The initial 
idea was to give to each citizen a smartcard containing her/his 
identification and insurance data. As smartcard storage ca- 
pacity increases, the information stored in the card can be 
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extended to the holder's doctors, emergency data (blood type, 
allergies, vaccination, etc.)* surgical operations, prescriptions, 
insurance data and even links to heavier data (e.g., X-ray exam- 
ination, scanner images, etc) stored on hospital servers. Dif- 
ferent users may query, modify; and create data in the holder's 
folder the doctors who consult the patient's past records and 
prescribe drugs, the surgeons who perform exams and opera- 
tions, the pharmacists who deliver drugs, the insurance agents 
who refund the patient, public organizations which maintain 
statistics or study the impact of drugs correlation in population 
samples, and finally the holder her/himself. 

We can easily observe that* (i) the amount of data is sig- 
nificant (more in terms of cardinality than in terms of volume 
because most data can be encoded); (ii) queries can be rather 
complex (e.g., a doctor asks for the last antibiotics prescribed 
to the patient); (iii) sophisticated access rights management 
using views and aggregate functions are highly required (e.g., 
a statistical organization may access aggregate values only but 
not the raw data); (iv) atomicity must be preserved (e.g., when 
the pharmacist delivers drugs); and (v) durability is manda- 
tory, without compromising data privacy (togged data stored 
outside the card must be protected). 

One may wonder whether the holder's health data ought 
to be stored in a smartcard or in a centralized database. The 
benefit of distributing the healthcare database on smartcards 
is threefold. First, health data must be made highly available 
(anywhere, anytime, on any terminal, and without requiring 
a network connection). Second, storing sensitive data on a 
centralized server may damage privacy. Third, maintaining a 
centralized database is fairly complex due to the variety of 
data sources. Assuming the health data is stored in the smart- 
card, the next question is why the aforementioned database 
capabilities need to be hosted in the smartcard rather than the 
terminals. The answer is again availability (the data must be 
exploited on any terminal) and privacy. Regarding privacy, 
since the data must be confined in the chip, so must the query 
engine and the view manager. As the smartcard is the unique 
trusted part of the system, access rights and transaction man- 
agement cannot be delegated to an untrusted terminal. 



3 Problem formulation 

In this section, we make clear the smartcard constraints in 
order to derive design rules for the PicoDBMS and state the 
problem. Our analysis is based on the characteristicsof both 



existing smartcard products and current prototypes [16, 39], 
and thus, should be valid fox a while. We also discuss how the 
main constraints of the smartcard will evolve in a near future. 



3.1 Smartcard constraints 

Current smaztcards include in a monolithic chip, a 32 bits RISC 
processor at about 30 MIPS, memory modules (of about 96 k£ 
of ROM, 4 kB of static RAM, and 1 2B kB ofEEPROM), secu- 
rity components (to prevent tampering), and take their electri- 
cal energy from the terminal [39]. ROM is used to store the op- 
erating system, the JavaCard virtual machine, fixed data, and 
standard routines. RAM is used as working memory for main- 
taining an execution stack and calculating results. EEPROM 
is used to store persistent information. EEPROM has very fast 
read time (60-100 ns/word) comparable to old-fashion RAM, 
but a dramatically slow write time (more man 1 ms/woxd). 

The main constraints of current smartcards are therefore: 
(i) the very limited storage capacity; (ii) the very slow write 
time in EEPROM; (iii) the extremely reduced size of the RAM; 
(iv) the lack of autonomy; and (v) a high security level that 
must be preserved in all situations. These constraints strongly 
distinguish smartcards from any other computing devices, in- 
cluding lightweight computers tike PDA. 

Let us now consider how hardware advances can impact on 
these constraints, in particular, memory size. Current smart- 
cards rely on a well-established and slightly out-of-date hard- 
ware technology (0.3 S /xm) in order to minimize the production 
cost (less than five dollars) and increase security [34]. Further- 
more, up to now, there was no real need for large memories 
in smartcard applications such as the holder's identification. 
According to major smartcard providers, the market pressure 
generated by emerging large storage demanding applications 
will lead to a rapid increase of the smartcard storage capac- 
ity. This evolution is however constrained by the smartcard 
tiny die size fixed to 25 mm 3 in the ISO standard [23], which 
pushes for more integration. This limited size is due to security 
considerations (to nunirnize the risk of physical attack [5]) and 
practical constraints (e.g., the chip should not break when the 
smartcard is flexed). Another solution to relax the storage limit 
is to extend the smartcard storage capacity with external mem- 
ory modules. This is being done by Gemplus which recently 
announced Ptnocchio [16], a smartcard equipped with 2MB 
of Flash memory linked to the microcontroller by a bus. Since 
hardware security can no longer be provided on this memory, 
its content must be either non-sensitive or encrypted 

Another important issue is the performance of stable mem- 
ory* Possible alternatives to the EEPROM are Flash memory 
and Ferroelectric RAM (FeRAM) [1 5] (see Table 2 for perfor- 
mance comparisons). Flash is more compact than EEPROM 
and represents a good candidate for high capacity smartcards 
[16]. However, Flash banks need to be erased before writing, 
which is extremely slow. This makes Flash memory appro- 
priate for applications with a high read/write ratio (e.g., ad- 
dress books). FeRAM is undoubtedly an interesting option for 
smartcards as read and write times are both fast. Although its 
theoretical foundation was set in the early 1950s, FeRAM is 
just emerging as an industrial solution. Therefore, FeRAM is 
expensive, less secure man EEPROM or Flash, and its integra- 
tion with traditional technologies (such as CPUs) remains an 
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Table 2. Performance of stable memories for the smartcard 



Memory type 



EEPROM FLASH 



FeRAM 



Read time (Avord) 60 to 150ns 70 to 200ns 150 to 200ns 



Write time (/word) 1 to 5 ms 

Erase time (/bank) Nose 

Lifetime (,) (/ceU) 10 5 write 
cycles 



StolOpts 150 to 200ns 

500tOo00ms None 

erase 10 10 to I0 ia 
cycles write cycles 



* A memory cell can be overwritten a finite number of time. 



methods of die PicoDBMS as well as the way transaction 
atomicity is achieved. 
• Non-autonomous: compared to other computers, the 
smartcard has no independent power supply, thereby pre- 
cluding disconnected and asynchronous processing. Thus, 
all transactions must be completed while the card is in- 
serted in a terminal (unlike PDA, write operations cannot 
be cached in RAM and reported on stable storage asyn- 
chronously). 



issue. Thus FeRAM could be considered a serious alternative 
only in the very long term [15]. 

Given these considerations, we assume in this paper 
a smartcard with a reasonable stable storage area (a few 
megabytes of EEPROM 2 ) and a small RAM area (some kilo- 
bytes). Indeed, there is no clear interest in having a large RAM 
area, given that the smartcard is not autonomous, thus pre- 
cluding asynchronous write operations. Moreover, more RAM 
means less EEPROM as the chip size is limited 



3.2 Impact on the PicoDBMS architecture 

We now analyze the impact of the smartcard constraints on 
the PicoDBMS architecture, thus justifying why traditional 
database techniques, and even lightweight DBMS techniques, 
are irrelevant The smartcard's properties and their impact are: 

• Highly secure: smartcard's hardware security makes it the 
ideal storage support forprivate data. The PicoDBMS must 
contribute to the data security by providing access right 
management and a view mechanism that allows complex 
view definitions (i.e., supporting data composition and ag- 
gregation). The PicoDBMS code must not present security 
holes due to the use of sophisticated algorithms 3 . 

• Highly portable: the smartcard is undoubtedly the most 
portable personal computer (the wallet computer). The 
data located on the smartcard are thus highly available. 
They are also highly vulnerable since the smartcard can 
be lost, stolen or accidentally destroyed The main conse- 
quence is that durability cannot be enforced locally. 

• Limited storage resources: despite the foreseen increase 
in storage capacity, the smartcard will remain the lightest 
representative of personal computers for a long time. This 
means that specific storage models and execution tech- 
niques must be devised to minimize the volume of per- 
sistent data (i.e., the database) and the memory consump- 
tion during execution. In addition, the functionalities of 
the PicoDBMS must be carefully selected and their im- 
plementation must be as light as possible. The lightest the 
PicoDBMS, the biggest the onboard database. 

• Stable storage is main memory: smartcard stable memory 
provides the read speed and direct access granularity of a 
main memory. Thus, a PicoDBMS can be considered as a 
main memory DBMS (MMDBMS), However the dramatic 
cost of writes distinguishes a PicoDBMS from a tradi- 
tional MMDBMS. This impacts on the storage and access 

2 Considering Flash instead of EEPROM will not change our con- 
clusions. It will just exacerbate diem. 
J Most security holes are the results of software bugs (34]. 



3.3 Problem statement 

To surnmarize, our goal is to design a PicoDBMS including 
the following components: 

« Storage manager: manages the storage of the database and 
the associated indices. 

• Query manager: processes query plans composed of se- 
lect, project, join, and aggregates. 

« Transaction manager: enforces the ACID properties and 
participates to distributed transactions. 

• Access right manager: provides access rights on base data 
and on complex user-defined views. 

Thus, the PicoDBMS hosted in the chip provides the min- 
imal subset of functionality that is strictly needed to manage 
in a secure way the data shared by all onboard applications. 
Other components (e.g., the GUI, a sort operator, etc.) can be 
hosted in the terminal or be dynamically downloaded when 
needed, without threatening security. In the rest of this pa- 
per, we concentrate on the components which require non- 
traditional techniques (storage manager, query manager, and 
transaction manager) and ignore the access right manager for 
which traditional techniques can be used 

When designing the PicoDBMS's components, we must 
follow several design rules derived from the smartcard's prop- 
erties: 

• Compactness rule: minimize die size of data structures 
and the PicoDBMS code to cope with the limited stable 
memory area (a few megabytes). 

• RAM rule: minimize the RAM usage given its extremely 
limited size (some kilobytes). 

• Write rule: minimize write operations given their dramatic 
cost (« 1 ms/word). 

• Read rule: take advantage of the fast read operations 
lOOns/word). 

• Access rule: take advantage of the low granularity and 
direct access capability of the stable memory for both read 
and write operations. 

• Security rule: never externalize private data from the chip 
and minimize the algorithms' complexity to avoid security 
holes. 



4 PicoDBMS storage model 

In this section, following the design rules for a PicoDBMS, we 
discuss the storage issues and propose a very compact model 
based on a combination of flat storage, domain storage, and 
ring storage. We also evaluate the storage cost of our storage 
model. 
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4 J Flat storage 

The simplest way to organize data is flat storage (FS), where 
tuples are stored sequentially and attribute values are embed- 
ded in the tuples. Although it does not impose it, the SCQL 
standard [24] considers FS as the reference storage model for 
smartcards. The main advantage of FS is access locality. How- 
ever, in our context, FS has two main drawbacks: 

• Space consuming: while normalization rules preclude at* 
tributes conjunction redundancy to occur, they do not 
avoid attribute value duplicates (e.g., the attribute Doc- 
tor.Specialty may contain many duplicates). 

• Inefficient, in the absence of index structures, all opera- 
tions are computed sequentially. While this is convenient 
for old fashion cards (some kilobytes of storage and a 
mono-relation select operator), this is no longer accept- 
able for future cards where storage capacity is likely to 
exceed 1 MB and queries can be rather complex. 

Adding index structures to FS may solve the second prob- 
lem while worsening the first one. Thus, FS alone is not ap- 
propriate for a PicoDBMS. 



42 Domain storage 

Based on the critique of FS, it follows that a PicoDBMS stor- 
age model should guarantee both data and index compactness. 
Let us first deal with data compactness. Since locality is no 
longer an issue in our context, pointer-based storage models 
inspired by MMDBMS [3 V 27, 31] can help reducing the data 
storage cost. The basic idea is to preclude any duplicate value 
from occuring. This can be achieved by grouping values in 
domains (sets of unique values). We call this model domain 
storage (DS). As shown in Fig. 1, tuples reference their at- 
tribute values by means of pointers. Furthermore, a domain 
can be shared among several attributes. This is particularly 
efficient for enumerated types, which vary on a small and de- 
termined set of values 4 . 

One may wonder about the cost of tuple creation, update, 
and deletion since they may generate insertion and deletion 
of values in domains. While these actions are more complex 
than their FS counterpart, their implementation remains more 
efficient in the smartcard context, simply because the amount 
of data to be written is much smaller. To amortize the slight 
overhead of domain storage, we only store by domain all large 
attributes (i.e., greater than a pointer size) containing dupli- 
cates. Obviously, attributes with no duplicates (e.g., keys) need 

4 Compression techniques can be advantageously used in conjunc- 
tion with DS to increase compactness [17]. 
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Fig. 2. Ring storage 



not be stored by domain but with FS. Variable-size attributes 
- generally larger than a pointer - can also be advantageously 
stored in domains even if they do not contain duplicates. The 
benefit is not storage savings but memory management sim- 
plicity (ail tuples of all relations become fixed-size) and log 
compactness (see Sect 6). 



4 J Ring storage 

We now address index compactness along with data compact- 
ness. Unlike disk-based DBMS that favor indices which pre- 
serve access locality, smartcards should make intensive use 
of secondary (i.e., pointer-based) indices. The issue here is to 
make these indices as compact as possible. Let us first consider 
select indices. A select index is typically made of two parts: a 
collection of values and a collection of pointers linking each 
value to all tuples sharing it. Assuming the indexed attribute 
varies on a domain, die index's collection of values can be 
saved since it exactly corresponds to the domain extension. 
The extra cost incurred by the index is then reduced to trie 
pointers linking index values to tuples. * 

Let us go one step further and get these pointers almost for 
free. The idea is to store these value-to-tuple pointers in place 
of the tuple-to-value pointers within the tuples (i.e., pointers 
stored in the tuples to reference their attribute values in the 
domains). This yields to an index structure which makes a ring 
from the domain values to the tuples. Hence, we call it ring 
index (see Fig. 2a). However, the ring index can also be used to 
access the domain values from the tuples and thus serve as data 
storage model. Thus we call ring storage (RS) the storage of 
a domain-based attribute indexed by a ring. The index storage 
cost is reduced to its lowest bound, that is, one pointer per 
domain value, whatever the cardinality of the indexed relation. 
This important storage saving is obtained at the price of extra 
work for projecting a tuple to the corresponding attribute since 
retrieving the value of a ring stored attribute means traversing 
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on average half of the ring (i.e., up to reaching the domain 
value). 

Join indices [40] can be treated in a similar way. A join 
predicate of the form (it a = 5.6) assumes that R.a and 5.6 
vary on the same domain. Storing both FLa and S.b by means 
of rings leads to defining a join index. In this way, each domain 
value is linked by two separate rings to all tuples from A and 
5 sharing the same join attribute value. However, most joins 
are performed on key attributes, FLa being a primary key and 
5.6 being the foreign key referencing ft.a. In our model, key 
attributes are not stored by domain but with FS. Nevertheless, 
since Ha is the primary key of R t its extension forms precisely 
a domain, even if not stored outside of R. Since attributes 5.6 
take their values in R.a*& domain, they reference Rjx values 
by means of pointers. Thus, the domain-based storage model 
naturally implements for free a unidirectional join index from 
5.6 to R.a (i.e., each 5 tuple is linked by a pointer to each 
R tuple matching with it). If traversals from R.a to 5.6 need 
to be optimized too, a bi-directional join index is required. 
This can be simply achieved by defining a ring index on 5.6. 
Figure 2b shows the resulting situation where each R tuple is 
linked by a ring to all 5 tuples matching with it and vice versa. 
The cost of a bi-directional join index is restricted to a single 
pointer per R tuple, whatever the cardinality of 5. Note that 
this situation resembles the well-known Codasyl model. 

4.4 Storage cost evaluation 

Our storage model combines FS, DS, and RS. Thus, the issue 
is to determine the best storage for each attribute. If the at- 
tributes need not be indexed, the choice is obviously between 
FS and DS. Otherwise, the choice is between RS and FS with a 
traditional index. Thus, we compare the storage cost for a sin- 
gle attribute, indexed or not, for each alternative. We introduce 
the following parameters: 

• CardRel: cardinality of the relation holding the attribute. 

• a: average length of the attribute (expressed in bytes). 

• p: pointer size (3 bytes will be required to address "large" 
memory of future cards). 

• 5: selectivity factor of the attribute. 5 = Card- 
DomlCardRel, where CardDom is the cardinality of the 
attribute domain extension (in all models). 5 measures the 
redundancy of the attribute (i.e., the same attribute value 
appears in 1/5 tuples). 

Cost(FS) = CardRe!*a II attribute storage cost in 

// the relation 

Cost(DS) = CardRel*p II attribute storage cost in 

// the relation 
+ S*CardRel*a II values storage cost in 
// the domain 

CostQndexedfS) = Cost(FS) II flat attribute storage cost 
+ S*CardReI*a II value storage cost in the 
//index 

+ CardReI*p II pointer storage cost in 
//the index 

Cost(RS) - Cost(DS) II domain-based attribute 

// storage cost 
+ S*CardRel*p II pointer storage cost in 
// the index 




(a)vmoutfrxtex 




(b)WMhtod0x 

Fig. 3. Storage models' tradeoff 



The cost equality between FS and DS gives : 5 = (o-p)/o. 
The cost equality between IndexedJS and RS gives: 

5 = a/p 

Figure 3a shows' the different values of 5 and a for which 
FS and DS are equivalent Thus, each curve divides the plan 
into a gain area for FS (above the curve) and a gain area for DS 
(under the curve). For values of a less than 3 (i.e., the size of a 
pointer), FS is obviously always more compact than DS. For 
higher values of a, DS becomes rapidly more compact than FS 
except for high values of 5. For instance, considering 5 = 0.5, 
that is the same value is shared by only two tuples, DS out- 
performs FS for all a larger than 6 bytes. The highera and the 
lower 5, the better DS. The benefit of DS is thus particularly 
important for enumerated type attributes. Figure 3b compares 
indexedJS with RS. The superiority of RS is obvious, except 
for 1- and 2-byte-long key attributes. Thus, Figs. 3a and 3b 
are guidelines for the database designer to decide how to store 
each attribute, by considering its size and selectivity. 



5 Query processing 

Traditional query processing strives to exploit large main 
memory for storing temporary data structures (e.g., hash ta- 
bles) and intermediate results. When main memory is not large 
enough to holdsome data, state-of-the-art algorithms (e.g., hy- 
brid hash join [33]) resort to materialization on disk to avoid 
memory overflow. These algorithms cannot be used for a Pi- 
coDBMS because: 

• Given the write rule and the lifetime of stable memory, 
writes in stable memory are proscribed, even for temporary 
materialization. 
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• Dedicating a specific RAM area does not help since we 
cannot estimate its size a priori. Making it small increases 
the risk of memory overflow, thereby leading to writes in 
stable memory. Making it large reduces the stable memory 
area, already limited in a smartcard (RAM rule). More- 
over, even a large RAM area cannot guarantee that query 
execution will not produce memory overflow [9]. 

• State-of-the-art algorithms are quite sophisticated, which 
precludes their implementation in a PicoDBMS whose 
code must be simple, compact, and secure (compactness 
and security rules). 

To solve this problem, we propose query processing tech- 
niques that do not use any working RAM area nor incur any 
writes in stable memory. In the following, we describe these 
techniques for simple and complex queries, including aggre- 
gation and remove duplicates. We show the effectiveness of 
our solution through a performance analysis. 



5. / Basic query execution without RAM 

We consider the execution of SPJ {Select/Project/Join) 
queries. Query processing is classically done in two steps. The 
query optimizer first generates an "optimal" query execution 
plan (QEF). The QEP is then executed by the query engine 
which implements an execution model and uses a library of 
relational operators [17], The optimizer can consider differ- 
ent shapes of QEP: left-deep, right-deep or bushy trees (see 
Fig. 4). In a left-deep tree, operators are executed sequentially 
and each intermediate result is materialized. On the contrary, 
right-deep trees execute operators in a pipeline fashion, thus 
avoiding intermediate result materialization. However, they 
require materializing in memory all left relations. Bushy trees 
offer opportunities to deal with the size of intermediate results 
and memory consumption [38]. 

In a PicoDBMS, the query optimizer should not consider 
any of these execution trees as they incur materialization. The 
solution is to only use pipelining with extreme right-deep trees 
where all the operators (including select) are pipelined. As left 
operands are always base relations, they are already materi- 
alized in stable memory, thus allowing us to execute a plan 
with no RAM consumption. Pipeline execution can be easily 
achieved using the well-known Ite vtor Model [17]. In this 
model, each operator is an iterator that supports three proce- 
dure calls; open to prepare an operator for producing an item, 
next to produce an item, and close to perform final clean-up. 
A QEP is activated starting at the root of the operator tree and 
progressing towards the leaves. The dataflow in the model is 
demand-driven: a child operator passes a tuple to its parent 
node in response toanerf call from the parent 

Let us now detail how select, project, and join are per- 
formed. These operators can be executed either sequentially 
or with a ring index. Given the access rule, the use of indices 
seems always to be the right choice. However, extreme right- 
deep trees allow us to speed-up a single select on the first base 
relation (e.g., Drug.type in our example), but using a ring in- 
dex on the other selected attributes (e.g., Visitdate) may slow 
down execution as the rings need to be traversed to retrieve 
their value. Project operators are pushed up to the tree since 
no materialization occurs. Note that the final project incurs 
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an additional cost in case of ring attributes. Without indices, 
joining relations is done by a nested-loop algorithm since no 
other join technique can be applied without ad hoc structures 
(e.g., hash tables) and/or working area (e.g., sorting). The cost 
of indexed joins depends on the way indices are traversed. 
Consider the indexed join between Doctor (ntuples) and Visit 
(rn tuples) on their key attribute. Assuming a unidirectional 
index, the join cost is proportional to n * m starting with Doc- 
tor and to m starting with Visit. Assuming now a bi-directional 
index, the join cost becomes proportional to n + ro starting 
with Doctor and to m 2 /2n starting with Visit (retrieving the 
doctor associated to each visit incurs traversing half of a ring 
in average). In the latter case, a naive nested loop join can be 
more efficient if the ring cardinality is greater than the tar- 
get relation cardinality (i.e., when m > n 3 ). In that case, the 
database designer must clearly choose a unidirectional index 
between the two relations. 



5.2 Complex query execution without RAM 

We now consider the execution of aggregate, sort, and du- 
plicate removal operators. At first glance, pipeline execution 
is not compatible with these operators which are classically 
performed oh materialized intermediate results. Such materi- 
alization cannot occur either in the smartcard due to the RAM 
rule or in the terminal due to the security rule. Note that sort- 
ing can be done in the terminal since the output order of the 
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result tuples is not significant, i.e., depends on the DBMS al- 
gorithms. 

We propose a solution to the above problem by exploiting 
two properties: (i) aggregate and duplicate removal can be 
done m pipeline if the incoming tuples are still grouped by 
distinct values; and (ii) pipeline operators are order-preserving 
since they consume (and produce) tuples in the arrival order. 
Thus, enforcing an adequate consumption order at the leaf of 
the execution tree allows pipelined aggregation and duplicate 
removal. For instance, the extreme right-deep tree of Fig. 4 
delivers die tuples naturally grouped by DrugM t thus allowing 
group queries on that attribute. 

Let us now consider query Q2 of Fig. 5. As pictured, exe- 
cuting Q2 in pipeline requires rearranging the execution tree 
so that relation Doctor is explored first Since Doctor contains 
distinct doctors, the tuples arriving to the count operator are 
naturally grouped by doctors. 

The case of Q3 is harder. As the data must be grouped 
by type of drugs rather than by Drug.id, an additional join is 
required between relation Drug and domain drug, type. Do- 
main values being unique, this join produces the tuples in the 
adequate order. If domain Drug.type does not exist, an opera- 
tor must be introduced to sort relation Drug in pipeline. This 
can be done by performing n passes on Drug where n is the 
number of distinct values of Drug.type, 



The case of Q4 is even trickier. The result must be grouped 
on two attributes (Doctor. id and Drugtype), introducing die 
need to start the tree with both relations! The solution is to 
insert a Cartesian product operator at the leaf of the tree in 
order to produce tuples ordered by Doctor. id and Drug.type. 
In this particular case, the query response time should be ap- 
proximately n times greater than the same query without the 
'group by* clause, where n is the number of distinct types of 
drugs. 

Q5 retrieves the distinct couples of doctor and type of pre- 
scribed drugs. This query can be made similar to Q4 by ex- 
pressing the distinct clause as an aggregate without function 
(i.e., the query "select distinct a\ , . . . , On from ..." is equiva- 
lent to ^select a\ , . , a n from . . . group by a\ , . , . ,On")- The 
unique difference is that the computation for a given group, 
i.e., (distinct result tuple) can stop as soon as one tuple has 
been produced. 



5 J Query optimization 

Heuristic optimization is attractive. However, well-known 
heuristics such as processing select and project first do not 
work here. Using extreme right-deep trees makes the former 
impractical and invalidates the tatter. Heuristics for join order- 
ing are even more risky considering our data structures. Con- 
versely, there are many arguments for an exhaustive search 
of the best plan. First, the search space is limited since: (i) 
there is a single algorithm for each operator, depending on the 
existing indices; (ii) only extreme right-deep trees are consid- 
ered; and (iii) typical queries will not involve many relations. 
Second, exhaustive search using depth-first algorithms do not 
consume any RAM. Finally, exhaustive algorithms are simple 
and compact (even if they iterate a lot). Under the assump- 
tion that query optimization is required in a PicoDBMS, the 
remarks above strongly argue in favor of an exhaustive search 
strategy. 



5 A Performance evaluation 

Our proposed query engine can handle fairly complex queries, 
taking advantage of the read and access rules 5 while satis- 
fying the compactness, write, RAM, and security rules. We 
now evaluate whether the PicoDBMS performance matches 
the smartcard application's requirements, that is, any query 
issued by the application can be performed in reasonable time 
(i.e., may not exceed the user's patience). Since the PicoDBMS 
code's simplicity is an important consideration to conform to 
the compactness and security rules, we must also evaluate 
which acceleration techniques (i.e., ring indices, query opti- 
mization) are really mandatory. For instance, an accelerator 
reducing the response time from 10 ms to 1 ms is useless in 
the smartcard context 6 . Thus, unlike traditional performance 
evaluation, our major concern is on absolute rather man rela- 
tive performance. 

5 With traditional DBMS, such techniques will induce so many 
disk accesses that the system would thrash! 

6 With traditional DBMS, such acceleration can improve the trans- 
actional throughput 
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Evaluating absolute response time is complex in the smart- 
card environment because all platform parameters (e.g., pro- 
cessor speed, caching strategy, RAM, and EEPROM speed) 
strongly impact on the measurements 7 . Measuring the per- 
formance of our PicoDBMS on Bull's smartcard technology 
is attractive but introduces two problems. First, Bull's smart- 
cards compatible with database applications are still proto- 
types [39]. Second, we are interested in providing the most 
general conclusions (i.e., as independent as possible of smart- 
card architectures). Therefore, we prefer to measure our query 
engine on two oldfashioned computers (a PC 486/25 Mhz and 
a Sun SparcStatton 1+) which we felt roughly similar to form- 
coming smartcard architectures. For each computer, we vary 
the system parameters (clock frequency, cache) and perform 
the experimentation tests. The performance ratios between 
all configurations were roughly constant (i.e., whatever the 
query), the slowest configuration (Intel 486 with no cache) per- 
forming eight times worse than the fastest (RISC with cache). 
In the following, we present response times for the slowest 
architecture to check the viability of our solutions in the worst 
environment 

We generated three instances of a simplified healthcare 
database: the small, medium, and large databases containing, 
respectively, (10, 30, 50) doctors, (100, 500, 1,000) visits, 
(300, 2,000, 5,000) prescriptions, and (40, 1 20, 200) drugs. Al- 
though we tested several queries, we describe below only the 
two mo st significant Query Ql, which contains three joins and 
two selects on Visit and Drug (with selectivities of 20% and 
5%), is representative of medium-complexity queries. Query 
Q4, which performs an aggregate on two attributes and re- 
quires the introduction of a Cartesian product, is representative 
of complex queries. For each query, we measure the perfor- 
mance for all possible query execution plans, excluding those 
which induce additional Cartesian product, varying the stor- 
age choices (with and without select and join ring indices). 
Figures 6 and 7 show the results for both best and worst plans 
on databases built with or without join indices. 

Considering SPJ queries, the PicoDBMS perfonnance 
clearly matches the application's requirements as soon as join 
rings are used. Indeed, the perfonnance with join rings is at 



7 With traditional DBMS, very slow disk access allows us to ignore 
finer parameters. 



most 146 ms for the largest database and with the worst execu- 
tion plan. With small databases, all the acceleration techniques 
can be discarded, while with larger ones, join rings remain nec- 
essary to obtain good response time. In that case, the absolute 
gain (1 10ms) between the best and the worst plan does not 
justify the use of a query optimizer. 

The performance of aggregate queries is clearly the worst 
because they introduce a Cartesian product at the leaf of the 
execution tree. Join rings are useful for medium and large 
databases. With large databases, the optimizer turns out to 
be necessary since the worst execution plan with join rings 
achieves a rather long response time (20.6 s). 

The influence of ring indices for selects (not shown) is in- 
significant Depending on the selectivity, it can bring slight im- 
provement or overhead on the results. Although it may achieve 
an important relative speed-up for the select itself, the abso- 
lute gain is not significant considering the small influence of 
select on the global query execution cost (which is not the case 
in disk-based DBMS). Select ring indices are, however, use- 
ful for queries with aggregates or duplicate removal, that can 
result in a join between a relation and the domain attribute. 
In that case, the select index plays the role of a join index, 
thereby generating a significant gam on large relations and 
large domains. 

Thus, this performance evaluation shows mat our approach 
is feasible and that join indices are mandatory in all cases 
while query optimization turns out to be useful only with large 
databases and complex queries. 



6 Transaction management 

Like any data server, a PicoDBMS must enforce the well- 
known transactional ACID properties [8] to guarantee the con- 
sistency of the local data it manages as well as be able to 
participate in distributed transactions. We discuss below these 
properties with respect to a PkoDBMS. 

• Atomicity: local atomicity means that the set of actions 
performed by the PicoDBMS on a transaction's behalf 
is made persistent following the oil or nothing scheme. 
Global atomicity: this means that all data servers - includ- 
ing the PicoDBMS - accessed by a distributed transaction 
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agree on the same transaction outcome (cither commit or 
rollback). The distinguishing features of a PicoDBMS re- 
garding atomicity are no demarcation between main mem- 
ory and persistent storage, the dramatic cost of writes, and 
the fact that they cannot be deferred. 

• Consistency, this property ensures that the actions per- 
formed by the PicoDBMS satisfy all integrity constraints 
defined on the local data. Considering that traditional in- 
tegrity constraint management can be used, we do not dis- 
cuss it any further. 

• Isolation: this property guarantees the serializability of 
concurrent executions. A PicoDBMS manages personal 
data and is typically single-user 8 . Furthermore, smartcard 
operating systems do not even support multithreading. 
Therefore, isolation is useless here. 

• Durability, durability means mat committed updates are 
never lost whatever the situation (i.e., even in case of a 
media failure). Durability cannot be enforced locally by 
the PicoDBMS because the smartcard is more likely to be 
stolen, lost or destroyed than a traditional computer, in- 
deed, mobility and smallness play against safety. Conse- 
quently, durability must be enforced through the network. 
The major issue is then preserving the privacy of data while 
delegating the durability to an external agent 

The remainder of this section addresses local atomicity, 
global atomicity, and durability. 



6. 1 Local atomicity 

There are basically two ways to perform updates in a DBMS. 
The updates are either performed on shadow objects that are 
atomically integrated in the database at commit time or done 
in place (i.e., the transaction updates the shared copy of the 
database objects) [8]. We discuss these two traditional models 
below. 

• Shadow update: This model is rarely employed in disk- 
based DBMSs because it destroys data locality on disk 
and increases concurrent updates on the catalog. In a Pi- 
coDBMS, disk locality and concurrency are not a concern. 
This model has been shown to be convenient for smart- 
cards equipped with a small Flash memory [25]. However, 
it is poorly adapted to pointer-based storage models like 
RS since the object location changes at every update. In 
addition, the cost incurred by shadowing grows with the 
memory size. Indeed, either the granularity of the shadow 
objects increases or the paths to be duplicated in the cata- 
log become longer. In both cases, the writing cost * which 
is the dominant factor - increases. 

• Update m-place: write-ahead logging (WAL) [8] is re- 
quired in this model to undo the effects of an aborted 
transaction. Unfortunately, the relative cost of WAL is 
much higher in a PicoDBMS than in a traditional disk- 
based DBMS which uses buffering to minimize I/Os. In a 
smartcard, the log must be written for each update since 
each update becomes immediately persistent This roughly 
doubles the cost of writing. 

' Even if the data managed by the PicoDBMS are shared among 
multiple users (e.g., as in the healthcare application), the PicoDBMS 
serves a single user at a time. 



Despite its drawbacks, update in-place is better suited than 
shadow update for a PicoDBMS because it accommodates 
pointer-based storage models and its cost is insensitive to the 
rapid growth of stable memory capacity. We also propose two 
optimizations to update in-place: 

• Pointer-based logging: traditional WAL logs die values of 
all modified data. RS allows a finer granularity by logging 
pointers in place of values. The smallest the log records, 
the cheapest the WAL. The logging process must consider 
two types of information: 

• Values: in case of a tuple update, the log record must con- 
tain the tuple address and the old attribute values, that is a 
pointer for all RS stored attributes and a regular value for 
FS stored attributes. In case of a tuple insertion or deletion, 
assuming each tuple header contains a status bit (Le., dead 
or alive), only the tuple address has to be logged in order 
to recover its state. 

• • Rings: tuple insertion, deletion, and update (of a ring at- 

tribute) modify the structure of each ring traversing the 
corresponding tuple L Since a ring is a circular chain of 
pointers, recovering its state means recovering the next 
pointer of t f s predecessor (let us call it tj^). The infor- 
mation to restore in t prtx i.next is either Vs address if t has 
been updated or deleted, or Lnext if t has been inserted, Vs 
address already belongs to the log (see above) and tnext 
does not have to be logged since Vs content still exists in 
stable storage at recovery time. The issue is how to iden- 
tify tprcd at recovery time. Logging this information can 
be saved at the price of traversing the whole ring starting 
from t, until reaching t again. Thus, ring recovery comes 
for free in terms of logging. 
• Garbage-collecting values: insertion and deletion of do- 
main values (domain values are never modified) should 
be logged as any other updates. This overhead can be 
avoided by implementing a deferred garbage collector that 
destroys all domain values no longer referenced by any tu- 
ple. Garbage-collecting a domain amounts to execute an 
ad hoc semi-join operator between the domain and all re- 
lations varying on it which discards the domain values that 
do not match 9 . The benefit of this solution is threefold: (i) 
the lazy deletion of unreferenced values does not entail the 
storage model coherency; (ii) garbage-collecting domain 
values is required anyway by RS (even in the absence of 
transaction control); and (iii) a deferred garbage-collector 
can be implemented without reference counters, thereby 
saving storage space. The deterred garbage collector can- 
not work in the background since smartcards do not yet 
support multi-threading. The most pragmatic solution is to 
launch it manually when the card is nearly full. An alter- 
native to this manual procedure is to execute the garbage 
collector automatically at each card connection on a very 
small subset of the database (so that its cost remains hid- 
den to the user). Garbage-collecting the database in such 
an incremental way is straightforward since domain values 
are examined one after the other. 



9 UnlikereachabUityalgorithmsthatsta^ 
and need marking [6], the proposed garbage-collector starts from the 
persistent leaves (i.e., the domain values) and exploits them one after 
the other, in a pipelined fashion (thus, it conforms to the RAM rule). 
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The update m-place model along with pointer-based log- 
ging and deferred garbage-collector reduces logging cost to its 
lowest bound, that is, a tuple address for inserted and deleted 
tuples, and the values of updated attributes (again, a pointer 
for DS and RS stored attributes). 



6.2 Globed atomicity 

Global atomicity is traditionally enforced by an atomic com- 
mitment protocol (ACP). The most well known and widely 
used ACP is 2PC [8]. While extensively studied ( 19] and stan- 
dardized [21, 29, 41], 2PC suffers from the following weak- 
nesses in our context: 

• Need for a standard prepared state: any server must ex- 
ternalize the standard Xa interface [41] to participate to 
2PC. Unfortunately, ISO defines a transactional interface 
for smartcards but it does not cover distributed transactions 
[24]. In addition, participating to 2PC requires building a 
local prepared state that consumes valuable resources. 

• Disconnection means aborting: a smaxtcard can be ex- 
tracted from its terminal or its mobile host (e.g., a cellular 
phone) can be temporarily unreachable during 2PC. A par- 
ticipant's disconnection leads 2PC to abort the transaction 
even if alt its operations have been successfully executed. 

• Badly adapted to moving participants: the 2PC incurs two 
message rounds to commit a transaction. Considering the 
high cost of wireless communication, the overhead is sig- 
nificant for mobile terminals equipped with a smartcard 
reader (e.g., PDA, cellular phones). 

As its name indicates, 2PC has two phases: the votingphzse 
and the decision phase. The voting phase is the means by 
which the coordinator checks whether or not the participants 
can locally guarantee the ACID properties of the distributed 
transaction. The decision is commit if all participants voteyer 
and abort otherwise. Thus, the voting phase introduces an 
uncertainty period at transaction termination that leads to the 
aforementioned drawbacks. 

Variations of one-phase commit protocols {IPC) have been 
recently proposed [2, 4, 35]. As stated in [2], IPC eliminates 
the voting phase of 2PC by enforcing the following properties 
on the participant's behavior (1) all operations are acknowl- 
edged before the IPC is launched; (2) there are no deferred 
integrity constraints; (3) all participants are ruled by a rigorous 
concurrency control scheduler; and (4) atl updates are logged 
on stable storage before IPC is launched. These assumptions 
guarantee, respectively, the A, C, I, D properties before the 
ACP is launched. Then, the ACP reduces to a single phase, that 
is broadcasting the coordinator's decision to all participants 
(this decision is commit if all transaction's operations have 
been successfully executed and abort otherwise). If a crash 
or a disconnection precludes a participant from conforrning 
to this decision, the corresponding transaction branch is sim- 
ply forward recovered (potentially at the next reconnectton). 
While the assumptions on the participant's behavior seem con- 
straining in the general case, they are quite acceptable in the 
smartcard context [10]. Property (I) is common to all ACPs 
and is enforced by the IS07S16 standard [22]; property (2) 
conforms to the fact that PicoDBMS have lighter capabilities 



than full-fledged DBMS; and property (3) is satisfied by def- 
inition since smartcards do not support parallel executions. 
Property (4) is discussed in Sect 63. 

Eliminating the voting phase of the ACP solves altogether 
the three aforementioned problems. However, one may won- 
der about the interoperability between transaction managers 
and data managers supporting different protocols (either IPC 
or 2 PC). We have shown in [1] that the participation of legacy 
(i.e., 2PC compliant) data managers in IPC is straightforward. 
Conversely, the participation of 1 PC compliant data managers 
(e.g., a smartcard) in the 2PC can be achieved by associat- 
ing a log agent to each participant. The role of the log agent 
is twofold. First, it manages the data manager's part of the 
IPC's coordinator log, forces it to stable storage during the 
2PC prepare phase, and exploits it if the transaction branch 
needs to be forward-recovered. Second, it translates the 2PC 
interface into that of I PC. The log agent can be located on the 
terminal, so that the benefit of 1 PC is lost for the terminal but 
it is preserved for the smartcard 



63 Durability 

Most IPC protocols assume that the coordinator is in charge 
of logging all participants' updates before triggering the ACP 
(all these protocols belong to the coordinator log family). Co- 
ordinator log [35] and implicit yes vote [4] assume that the 
participants piggyback their log records on the acknowledg- 
ment messages of each operation while coordinator logical 
log [2] assumes that the coordinator logs all operations sent to 
each participant In all cases, the durability of the distributed 
transaction relies on the coordinator log. Thus, 1 PC is a means 
by which global atomicity and durability can be solved alto- 
gether, at the same price. 

Two issues remain to be solved: (i) where to store the 
coordinator log; and (ii) how to preserve the security rule, 
that is, how to make the log content as secure as the data 
stored in the smartcard. Since the log must sustain any kind 
of failure, it must be stored on the network by a trustee server 
(e.g. t a public organism, a central bank, the card issuer, etc.). 
If some transactions are executed in disconnected mode (e.g., 
on a mobile terminal), the durability will be effective only at 
the time the terminal reconnects to the network. Protecting 
the log content against attacks imposes encryption. The way 
encryption is performed depends on the model of logging. 
If the coordinator log is fed by the log records piggybacked 
by the participants, the smartcard can encrypt them with an 
algorithm based on a private key (e.g., DES [28]). Otherwise 
(i.e., if the coordinator logical log scheme is selected), the 
smartcard can provide the coordinator with a public key that 
will be used by the coordinator itself to encrypt its tog [32]. 



6.4 Transaction cost evaluation 

The goal of this section is to approximate the time required by 
a representative update transaction. The objective is to confirm 
whether or not the write performance of smartcards assumed 
in this paper is acceptable for database applications like health 
cards. To this end, we estimate the time required to create a 
tuple in a relation, including the creation of domain values, 
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the insertion of the tuple in the rings potentially defined on 
this relation and the tog time. Let us introduce the following 
parameters, in addition to those already defined in Sect 4.4: 

• nbAttFS: number of FS stored attributes 

• nbAttDS: number of DS stored attributes 

• nbAttRS: number of RS stored attributes 

• to: size of a word (4 bytes in a 32-bit card) 

• t: time to write one word in stable storage 

(5 ms in the worst case) 

Cost{inserfluple) = 

a(nbAttFS*a + nbAttDS*p + nbAttRS ^p^w] // <D 

+ (nbAttRS + nbAttDS) • S • [aAv] //<2> 

+ nbAttRS *[p/w] II® 

+ [p/w] //<S> 

)*t //<§> 

<S> Tuple size 

® Domain values size. S « probability 

to create a new domain value 
@ Ring pointers to be updated 
CD Log record size 
@ Write time 

Let us consider a representative transaction executed on 
the healthcare. This transaction inserts a new tuple in Doctor 
and Visit and five tuples in Prescription and Drug. This is 
somehow a worst case for this application in the sense that the 
visited doctor is a new one and prescribes five new drugs. The 
considered attribute distribution is as follows: . 



Doctor (nbAttFS=3, nbAttDS=4> nbAnRS=0), 

Visit (nbAttFS=2, nbAt(DS=3, nbAttRS=2), 

Prescription (nbAttFS-J. nbAttDS~l t nbAttRS=2), 

Drug (nbAttFS-2, nbAttDS=4 t nbAttRS=0). 



The average attribute length a is fixed to 1 0 bytes. Figure 8 
plots the update transaction execution time depending on S 
(5 = 0 means that all attribute values already exist in the 
domains, while 5 = 1 means that all these values need be 
inserted in the domains). 

The figure is self-explanatory. Note that the logging cost 
represents less than 3% of the total cost. This simple analysis 
shows that the time expected for this kind of transaction (less 
than 1 s)is clearly compatible with the healthcare application's 
requirements. 



7 Conclusion 

As smartcards become more and more versatile, multi- 
application, and powerful, the need for database techniques 
arises. However, smartcards have severe hardware limita- 
tions which make traditional database technology irrelevant. 
The major problem is scaling down database techniques so 
they perform well under these limitations. In this paper, we 
addressed this problem and proposed the design of a Pi- 
coDBMS, concentrating on the components which require 
non-traditional techniques (storage manager, query manager, 
and transaction manager). 

This paper makes several contributions. First, we an- 
alyzed the requirements for a PicoDBMS based on a 
healthcare application which is representative of personal 
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Fig. 8. Performance of a typical update transaction 

folder applications and has strong database requirements. 
We showed that the minimal functionality should include 
select/project/join/aggregate, access right management, and 
views as well as transaction's atomicity and durability. 

Second, we gave an in-depth analysis of the problem by 
considering the smartcard hardware trends. Based on this anal- 
ysis, we assumed a smartcard with a reasonable stable memory 
. of a few megabytes and a small RAM of some kilobytes, and 
we derived design rules for a PicoDBMS architecture. . 

Third, we proposed a new highly compact storage model 
that combines flat storage (FS), domain storage (DS), and ring 
storage (RS). Ring storage reduces the indexing cost to its 
lowest bound. Based on performance evaluation, we derived 
guidelines to decide the best way to store an attribute. 

Fourth, we proposed query processing techniques which 
handle complex query plans with no RAM consumption. This 
is achieved by considering extreme right-deep trees which can 
pipeline all operators of the plan including aggregates. We 
also argued that, if query optimization is needed, the strategy 
should be exhaustive search. We measured the performance of 
our execution model with an implementation of our query en- 
gine on two old-fashioned computers which we configured to 
be similar to forthcoming smartcard architectures. We showed 
that the resulting performance matches the smartcard applica- 
tion's requirements. 

Finally, we proposed techniques for transaction atomic- 
ity and durability. Local atomicity is achieved through up- 
date in-place with two optimizations which exploit the stor- 
age model: pointer-based logging and garbage collection of 
domain values. Global atomicity and durability are enforced 
by IPC which is easily applicable in the smartcard context 
and more efficient than 2PC. We showed that the performance 
of typical update transactions is acceptable for representative 
applications like the health card. 

This work is done in the context of a new project with 
Bull Smart Cards and Terminals. The next step is to port our 
PicoDBMS prototype on Bull's smartcard new technology, 
called OverSoft [12], and to assess its functionality and per- 
formance on real-world applications. To this end, a bench- 
mark dedicated to PicoDBMS must be set up. We also plan to 
address open issues such as protected logging for durability, 
query execution on encrypted data (e.g., stored in an externa] 
Flash), and statistics maintenance on a population of cards. 
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1. Introduction 

One of the primary uses today for computer technol- 
ogy is information storage and retrieval Typical search- 
ing applications include dictionaries, telephone listings, 
medical databases, symbol tables for compilers, and 
storing a company's business records. Each package of 
information is stored in computer memory as a record. 
We assume there is a special field in each record, called 
the key, that uniquely identifies it The job of a cgnrehtng 
algorithm is to take an input K and return the record (if 
any) that has if as its key. 

Noshing is a widely used searching technique because 
no matter how many records are stored, die average 
search times remain bounded. The common element of 
all hashing algorithms is a predefined and quickly com- 
puted hash Junction 

hash: {all possible keys} {1, X . . . . Af } 

that assigns each record to a hash address in a uniform 
manner. (The problem of designing hash functions that 
justify this assumption, even when the distribution of the 
keys is highly biased, is well-studied [7, 2].) Hashing 
methods differ from one another by how they resolve a 
collision when the hash address of the record to be 
inserted is already occupied. 

This paper investigates the coalesced hashing algo- 
rithm, which was first published 22 years ago and is still 
one of the faster known searching methods (16, 7]. The 
total number of available storage locations is assumed to 
be fixed It is also convenient to assume that these 
locations are contiguous in memory. For the purpose of 
notation, we shall number the hash table slots 1,2,..., 
Ml. The first M slots, which serve as the range of the 
hash function, constitute the address region. The remain- 
ing M* — M slots are devoted solely to storing records 
that collide when inserted; they are called the cellar. 
Once the cellar becomes full, subsequent colliders must 
be stored in empty slots in the address region and, thus, 
may trigger more collisions with records inserted later. 

For this reason, the search performance of the coa- 
lesced hashing algorithm is very sensitive to the relative 
sizes of the address region and cellar* In Sec 4, we apply 
the analytic results derived in [10, 11, 13] in order to 
optimize the ratio of their sizes, fi — M/M\ which we 
call the address factor. The optimizations are based on 
two performance measures: the number of probes per 
search and the running time of assembly language ver- 
sions. There is no unique best choice for #— the optimum 
address factor depends on the type of search, the number 
of inserted records, and the performance measure cho- 
sen—but we shall see that the compromise choice p ~ 
0.86 works well in many situations. The method can be 
further turned to meet specific needs. 

Section 5 shows that this tuned method dominates 
several popular hashing algorithms including standard 
coalesced hashing (in which fi = IX separate (or direct) 
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chaining, linear probing, and double hashing The last 
thicc sections deal with variation* and different imple- 
mentations for coalesced hashing including deletion al- 
gorithms, alternative insertion methods and external 
searching on secondary storage devices. 

This paper is designed to provide a comprehensive 
treatment of the many practical issues concerned with 
the implementation of the coalesced hashing method. 
Readers interested in the theoretical justification of the 
results in this paper can consult {10, 11, 13. 14, 1]* 



2. The Coalesced Hashing Algorithm 

The algorithm works like this: Given a record with 
key the algorithm searches for it in the hash table, 
starting at location hash(K) and following the links in 
the chain. If the record is present in the table, then it is 
found and the search is successful', otherwise, the end of 
the chain is reached and the search is unsuccessful. For 
simplicity, wc assume that the record is inserted when- 
ever the search ends unsuccessfully, according to the 
following rule: If position hash(K) is empty, then the 
record is stored at that location; else, it is placed in the 
largest-numbered empty slot in the table and is linked to 
the end of the chain. This has the effect of potting the 
first M' — M colliders into the cellar. 

Coalesced hashing is a generalization of the well- 
known separate (or direct) chaining method. The sepa- 
rate chaining method halts with overflow when there is 
no more room in the cellar to store a collider. The 
example in Fig. 1(a) can be considered to be an example 



of tvtfh rrta\+*r+A hiriwng and «*parat» rhamjn^ >w-ftn<^ 

the cellar is large enough to store the three colliders. 

Figures 1(b) and 1(c) show how the two methods 
differ. The cellar contains only one slot in the example 
in Fig. 1(b). When the key mark collides with donna at 
slot 4, the cellar is already fiilL Separate Naming would 
report overflow at this point The craW-snad hashing 
method, however, stores the key mark in the largest- 
numbered empty space (which is location 10 in the 
address region). This causes a later collision when dave 
hashes to position 10, so davb is placed in slot 8 at the 
end of the chain containing' donna and mark. The 
method derives its name from this "coalescing" of rec- 
ords with different hash addresses into single chains. 

The average number of probes per search shows 
marked improvement in Fig. 1(b), even though coalesc- 
ing has occurred. Intuitively, the larger address region 
spreads out the records more, evenly and causes fewer 
collisions, Le n the hash function can be thought of as 
"shooting" at a bigger target The cellar is now too small 
to store these fewer colliders, so it overflows. Fortunately, 
this overflow occurs late in the game, and the pileup 
phenomenon of coalescing is not significant enough to 
counteract the benefits of a larger address region. How- 
ever, in the extreme case when M AT ■» 11 and there 
is no cellar (which we call standard coalesced hashing), 
coalescing begins too early and search time worsens (as 
typified by Figure 1(c)). Determining the optimum ad- 
dress factor ft - MjM' is a major focus of this paper. 

The first order of business before we can start a 
detailed study of the coalesced hashing method is to 
formalize the algorithm and to define reasonable 
measures of seaix& performance. Let us assume that each 



Fig. 1. Coalesced hashing, A/' - II, JT- 8. The sizes of the address region arc (») A* • ft, (b)A/- 10, and (c) H - li. 
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average § probes per successful search: (a) 12/8 = 1.5, (b) 11/6 = 1.375, (c) 14/8 = 1.75. 
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of the M* contiguous dots in the coalesced hash table 
has the following organization: 



E 








hi 
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KEY 


other fields 


LINK 
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Y 









For each value of i between 1 and M EMPTY [i] is a 
one-bit field that denotes whether the tth slot is unused, 
KEY[Q stores the key Of any), and LINK[i] is either the 
index to the next spot in the chain or else the null value 
0. 

The algorithms in this article are written in the 
English-hie style used by Knuth in order to make them 
readily understandable to all and to facilitate compari- 
sons with the algorithms contained in (7, 4, 12). Block- 
structured languages, like PL/I and Pascal, are good for 
expressing complicated program modules; however, they 
are not used here, because hashing algorithms are so 
short that there is no reason to discriminate against those 
who are not comfortable with such languages. 

Algorithm C (Coalesced hashing search and insertion). 
This algorithm searches an Af'-siot hash table, looking 
for a given key K. If the search is unsuccessful and the 
table is not full, then K is inserted. 

The size of the address region is M ; the hash function 
hash returns a value between 1 and M (inclusive). For 
convenience, we make use of slot 0, which is always 
empty. The global variable R is used to find an empty 
space whenever a collision must be stored in the table. 
Initially, the table is empty, and we have R « M' + I; 
when an empty space is requested, R is decremented 
until one is found. We assume that the following initial- 
izations have been made before any searches or inser- 
tions are performed: AS «— \&M% for some constant 
0 < fi 2= 1; EMPTY[Q «- «™e, for all 0 S i * M '; and 
R+-M' + h 

CI. [Hash.] Set i «- hash(Ky (Now 1 ^ i < Af.) 

G2. [Is there a chain?] \fEMPTY\i\ then go to step C6. 
(Otherwise, the ith slot is occupied, so we will look 
at the chain of records that starts there.) 

C3, [Compare.] \fK- KEY[t) t the algorithm terminates 
successfully. 

C4. [Advance to next record.] If LlNK[i] * 0, then set 
i £JNK[(] and go back to step C3. 

CS. [Find empty slot] (The search for K in the chain 
was unsuccessful, so we will try to find an empty 
table slot to store K.) Decrease R one or more times 
until EMPTY[R) becomes true. If R - 0, then there 
are no more empty slots, and the algorithm termi- 
nates with overflow. Otherwise, append the *th cell 
to the chain by setting UNK[i\ «- R; then set / «- 
R- 

C6. [Insert new record.] SetEMPTYlI}+-M$e>KEY[(} 
«- K f UNK[i] «- 0, and initialize the other fields in 
the record ■ 
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In this paper, we concern ourselves with measuring 
the searching phase of Algorithm C and ignore for the 
most part the insertion time in steps C5 and Co. (The 
time for step C5 is not significant, because the total 
number of times R is decremented over the course of aO 
the tw«gr«tnnft cannot be more than the number of in* 
sorted records; hence, the amortized expected number of 
decrements is at most I . Hie decrementing operation can 
also be done in parallel with steps C1-C4.) Our primary 
measure of search performance is the number of probes 
per search* which is the number of different table slots 
that are accessed while searching. In Algorithm C, this 
quantity is equal to 

max{l, number of times step C3 is performed) 

For example, in Fig. 1(b), the unsuccessful searches for 
keys a.l. and tootie (immediately prior to their inser- 
tions) each took one probe, while a successful search for 
dave would take two probes. 

The average performance of the algorithm is ob- 
tained by assuming that all searches and insertions are 
random. The Appendix contains a discussion of the 
probability model as well as the formulas for the ex- 
pected number of probes in unsuccessful and successful 
searches. 



3. Assembly Language Implementation 

Even though probe-counting gives us a good idea of 
search performance, other factors (such as the complexity 
of the search loop and the overhead is computing the 
hash address) also, affect the running time when Algo- 
rithm C is programmed for a real computer. For com- 
pleteness, we optimize the running time of assembly 
language versions of coalesced hashing. 

We choose to program in assembly language rather 
than in some high-level language like Fortran, PL/I, or 
Pascal, in order to achieve maximum possible efficiency. 
Top efficiency is important in large-scale applications of 
hashing, but it can also be achieved in smaller systems 
with little extra effort, because hashing algorithms are so 
short that implementing them (even in assembly lan- 
guage) is easy. We use a hypothetical language based on 
Knuth* s mix [6] because its features are similar to most 
well-known machines and its inherent simplicity allows 
us to write programs in clear and concise form. 

Program C below is a Mix-like implementation of 
Algorithm C. liberties have been taken with the lan- 
guage for purposes of clarity; the actual mix code appears 
in [10]. Hie program is written in a five-column format: 
the first column gives the line numbers, the second 
column lists the instruction labels, the third column 
contains the assembly language instructions, the fourth 
column counts the number of times the instructions are 
executed, and the last column is for comments that 
explain what the instructions do. The syntax of the 
commands should be clear to those familiar with assem- 
bly language programming. The four memory registers 
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used in Program C ore named r A, rX, rl, and rJ. The 
reference K£Y(I) denotes the contents of the memory 
location whose address is the value of KEY phis the 
contents of rl. {This is KEY[i] in the notation of Algo- 
rithm C.) 

Program C (Coalesced hashing search and insertion). 
This program follows the conventions of Algorithm C, 
except that the EMPTY Held b implicit in the LINK 



field: empty slots are marked by a —I in the LINK field 
of that slot Null links are denoted by a 0 in the LINK 
field. The variable R and the key K are stored in memory 
locations R and K. Registers rl and r A are used to store 
the values of / and K. Register rJ stores either the value 
of LINK[l] or It The instruction labels SUCCESS and 
OVERFLOW are for exiting and are assumed to lie 
somewhere outside this code. 



01 


START 


LD 




1 


Step CI. Load rX with AT. 


02 




ENT 


A.0 


[ 


Enter 0 into rA. 


03 




DIV 


»M- 


I 


rA *-l*/A/J t tX+-K mod M. 


04 




ENT 


I.X ' 


1 


Enter rX into rl. 


05 




INC 


I, 1 


j 


Increment rl by L 


06 




LD 


A, K 


1 


Load r A with JC 


07 




LD 


J,UNK(I) 


1 , 


Step C2. Load rJ with UNK[i]. - 


08 




JN 


J, STEP6 


1 


Jump to STEP6 if UNK[i] < 0. 


09 




CMP 


A, KEY(I) 


A 


Step CJ. Compare K with KEY[il 


10 




JE 


SUCCESS 


A 


Exit (successfully) if K « KEY[il 


II 




JZ 


J, STEPS 


A-S\ 


Jump to STEPS \£UNK[t] = 0, 


12 


STEP4 


ENT 


I, J 


C- 1 


Step C4. Enter rJ into rL 


13 




CMP 


A. KEY(1) 


C- 1 


Step C3. Compare K with KEY[Q. 


14 




JE 


SUCCESS 


C- 1 


Exit (successessrully) if K - KEY[i\ 


15 




LD 


J,LINK(I) 


C- 1-S2 


Load rJ with LINK[}\ 


16 




JNZ 


J.STEP4 


C- 1 -52 


Jump to STEP4 if LINK[i) * 0. 


17 


STEPS 


LD 


J,R 


A - 5 


Step C5. Load rJ with R. 


18 




DEC 


J.I 


r 


Decrement R by 1. 


19 




LD 


X, UNK(J) 


T 


Load rX with LINK[R}. 


20 




JNN 




T 


Go back two steps if LINK[R] > 0. 


21 




JZ 


J, OVERFLOW 


A-S 


Exit (with overflow) if R ~ 0. 


22 




ST 


J, UNK(!) 


A~S 


Store R in LINK[i] 


23 




ENT 


M 


A~S 


Enter rJ into rl. 


24 




ST 


J,R 


A-S 


Update R in memory. 


25 


STEP6 


ST 


0, LINKfl) 


1-5 


Step C6. Store 0 in UNK[i]. 


26 




ST 


A, KEY(I) 


1 - 5 


Store Ku KEY\i].m 



The execution time is measured in mix units of time, 
which we denote u. The number of time units required 
by an instruction is equal to the number of memory 
references (including the reference to the instruction 
itself)- Hence, the LD, ST, and CMP instructions each 
take two uniu of time, while ENT, INC, DEC, and the 
jump instructions require only one time unit. The divi- 
sion operation used to compute the hash address is an 
exception to this rule; it takes I4w to execute. 

The running time of a mix program is the weighted 
sum 



each latntcUim 
in Uw pntnn 



/ 1 

[the 

\ is 



# times \/ # time units \ . 
instruction II required by ] (1) 
executed /\ the instruction/ 



This is a somewhat simplistic model, since it does not 
make use of cache or bu tiered memory for fast access of 
frequently used data, and since it ignores any interven- 
tion by the operating system. But it places all hashing 
algorithms on an equal footing and gives a good indi- 
cation of relative merit. 
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Hie fourth column of Program C expresses the num- 
ber of times each instruction is executed in terms of the 
quantities 

C = number of probes per search. 

A =» 1 if the initial probe found an occupied slot, 

0 otherwise. 
S * I if successful, 0 if unsuccessful 
T « number of slots probed while looking for an empty 

space. 

We further decompose S into 51 + 52, where Si » 1 if 
the search is successful on the first probe, and 51 « 0 
otherwise. By formula (1), the total running time of the 
searching phase is 

(7C + 4A + 17 - 35 + 2SX)u (2) 

and the insertion of a new record after an unsuccessful 
search (when 5«0) takes an additional (8*4 + 4r + 4)w. 
The average running time is the expected value of (2), 
assuming that all insertions and searches are random. 
The formula can be obtained by replacing the variables 
in Eq. (2) with their expected values. 
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4> Tuning £ to Obtain Optimum Performance 

The purpose of the analysis in [10, II, 13] ts to show 
how the average-case performance of the coalesced hash- 
ing method varies as a function of the address factor fi 
- M/AT and the load factor a ~ N/M'. In this section, 
for each fixed value of a, we make use of those results in 
order to "tune" our choice of/8 and speed up the search 
times. Our two measures of performance are die expected 
number of probes per search and the average running 
time of assembly language versions. In the latter case, 
we study a mix implementation in detail, and then show 
how to apply what we learn to other assembly languages. 

Unfortunately, there is no single choice of fi that 
yields best results; the optimum choice /Soft is a function 
of the load factor a and it is even different for unsuc- 
cessful and successful searches. The section concludes 
with practical tips on how to initialize fi. In particular, 
we shall see that the choice fi = 0.86 works well in most 
situations. 

4J Number of Probes Per Search 

For each fixed value of a, we want to find the values 
fiorr that minimize the expected number of search probes 
in unsuccessful and successful searches. Formulas (Al) 
and (A2) in the Appendix express the average number 
of probes per search as a function of three variables: the 
load factor a = N/M the address factor fi « M/M\ 
and a new variable A * L/M, where L is the expected 
number of inserted records needed to make the cellar 
become fulL The variables fi and A are related by the 
formula 

e-> + A = I (3) 

Formulas (Ai) and (A2) each have two cases, "a < 
Xfi" and **a > Xfi? which have the following intuitive 
meanings: The condition a < Xfi means that with high 
probability not enough records have been inserted to fill 
up the cellar, while the condition a > Xfi means that 
enough records have been inserted to make the cellar 
almost surely full. 

The optimum address factor fiorr is always located 
somewhere in the "a > Xfi" region, as shown in the 
Appendix. The rest of the optimization procedure is a 
straightforward application of differential calculus. First, 
we substitute Eq. (3) into the "a * Xfi" cases of the 
formulas for the expected number of probes per search 
in order to express them in terms of only the two 
variables a and A. For each nonzero fixed value of a, the 
formulas are convex w.r.t. A and have unique minima. 
We minimize them by setting their derivatives equal to 
0. Numerical analysis techniques are used to solve the 
resulting equations and to get the optimum values of A 
for several different values of a. Then we reapply Eq. (3) 
to express the optimum points in terms of fi. The results 
are graphed in Fig. 2(a), using spline interpolation to fill 
in the gaps. 



4l2 mix Running Times 

Optimizing the mtx execution times could be tricky, 
in general, because the formulas might have local as well 
as global minima. Then when we set the derivatives 
equal to 0 in order to find fiorr, there might be several 
roots to the resulting equations. The crucial fact that lets 
us apply the same optimization techniques we used above 
for die number of probes is that the formulas for the mix 
running times are well-behaved, as shown in the Appen- 
dix. By that we mean that each formula is minim'™** at 
a unique Popt, which occurs either at the endpoint a « 
Xfi or at the unique point in the **a 3s Xfi n region where 
the derivative w*r.t. fi is 0. 

The optimization procedure is the same as before. 
Hie expected values of formulas (A4) and (A3), which 
give the mix ninning times for unsuccessful and success- 
ful searches, are functions of the three variables a, fi, and 
A. We substitute Eq. (3) into the expected running times 
in order to express fi in terms of A. For several different 
load factors a and for each type of search, we find the 
value of A that minimizes the formula, and then we 
retranslate this value via Eq. (3) to get /fopr. Figure 2(b) 
graphs these optimum values fiorr as a function of a; 
spline interpolation was used to fill in the gaps. As in the 
previous section, the formulas for the average unsuccess- 
ful and successful search times yield different optimum 
address factors. For the successful search case, notice 
how closely fiopr agrees with the corresponding values 
that minimize the expected number of probes. 



Fig. 2. The values fiopr thai optimize search performance for (he 
following three measures: (a) the rupcrtcd number of probes per 
ce&rch, (b) the ex p ect e d running time of Program G» and (c) tbe 
expected assembly language running lime for targe keys. 
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43 Applying the Results to Other implementations 

Our mix analysts suggests two important principles 
to be used in finding fiopr for a particular implementa* 
lion of coalesced hashing. First* the formulas for the 
expected number of times each instruction in the pro- 
gram is executed (which are expressed for Program C in 
terms of CM, £. S 1, 52, and T) may have the two cases, 
*Vr £ \fi" and "a s \fi" but probably not more. 

Second, the same optimization process as above can 
be used to find /Wr, because the formulas for the 
running times should be well-behaved for the following 
reason: The main difference between Program C and 
another implementation is likely to be the relative time 
it takes to process each key. (The keys are assumed to be 
very small in the mix version.) Thus, the unsuccessful 
search time for another implementation might be ap- 
proximately 

[(2* + 5)C + (2« + 2)A + (-2jc + t9)K (4) 

where u' is the standard unit of time on the other 
computer and tc is bow many times longer it takes to 
process a key (multiplied by Successful search 

times would be about 

[(2k + 5)C+ 18 + 2S1>i' (5) 

Formulas (4) and (5) were calculated by increasing the 
execution times of the key-processing steps 9 and 13 in 
Program C by a factor of *. (See formulas (A4) and (A5) 
for the * * 1 case.) We ignore the extra time it takes to 
load the larger key and to compute the hash function, 
since that does not affect the optimization. 

The role of C in formula (4) is less prevalent than in 
(A4) as k gets large: the ratio of the coefficients of C and 
A decreases from 7/4 in (A4) and approaches the limit 
2/2 =* 1 in formula (4). Even in this extreme case, 
however, computer calculations show that the formula 
for the average running time is well-behaved The values 
of /fopT that minimize formula (4) when « is large are 
graphed in Fig. 2(c). 

For successful searches, however, the value of C more 
strongly dominates the running times for larger values of 
«, so the limiting values of flotT in Fig. 2(c) coincide with 
the ones that minimize the expected number of probes 
per search in Fig. 2(a). Figure 2(b) shows that the 
approximation is close even for the case k - 1, which is 
Program C 

4.4 How to Choose fi 

It is important to remember that the address region 
size M « l&MI must be initialized when the hash table 
is empty and cannot change thereafter. Unfortunately, 
the last two sections show that each different load factor . 
a requires a different optimum address factor porr\ in 
fact, the values offiotr differ for unsuccessful and suc- 
cessful searches. This means that optimizing the average 
unsuccessful (or successful) search time for a certain load 
factor a will lead to suboptimum performance when the 
load factor is not equal to a. 

9X6 



One strategy is to pick fi = 0.782, which 
the expected number of probes per unsuccessful search 
as well as the average mix unsuccessful search time when 
the table is full (Le., load factor a « 1), as indicated in 
Fig. 2. This choice of fi yields the best absolute bound 
on search performance, because when the table is full, 
search times are greatest and unsuccessful searches av- 
erage slightly longer than successful ones. Regardless of 
the load factor, the expected number of probes per search 
would be at most 1.79, and the average mix searching 
time would be bounded by 33J2u. 

Another strategy is to pick some compromise address 
factor that leads to good overall performance for a large 
range of load factors. A reasonable choice is fi = 0.86; 
then the unsuccessful searches are optimized (over all 
other values of 0) when the bad factor is s&68 (number 
of probes) and ^0-56 (mix), and the successful search 
performance is optimized at load factors s0.94 (number 
of probes) and =0.95 (mix). 

Figures 3 through 6 graph the expected search per- 
formance of coalesced hashing as a function of a for 
both types of searches (unsuccessful and successful) and 
for both measures of performance (number of probes 
and mix running time). The Ci curve corresponds to 
standard coalesced hashing (i.e., fi ■» t); the Cue line is 
our compromise choice fi — 0.86; and the dashed tine 
Copt represents the best possible search performance 
that could be achieved by tuning (in which fi is optimized 
for each load factor). 

Notice that the value fi « 0.86 yields near-optimum 
search times once the table gets half-full, so this compro- 
mise offers a viable strategy. Of course, if some prior 
knowledge about the types and frequencies of the 
searches were available, we could tailor our choice of/? 
to meet those specific needs. 



5. Comparisons 

In this section, we compare the searching times of the 
coalesced hashing method with those from a represent- 
ative collection of hashing schemes: standard coalesced 
hashing (Ci), separate chaining (S), separate chaining 
with ordered chains (SO), linear probing (L), and double 
hashing (D). Implementations of the methods are given 
in (10]. 

These methods were chosen because they are the 
most well-known and since they each have implemen- 
tations similar to that of Algorithm C. Our comparisons 
are based both on the expected number of probes per 
search as well as on the average mix running time. 
Coalesced hashing performs better than the other 
methods. The differences are not so dramatic with the 
Mix search times as with the number of probes per 
search, due to the large overhead in computing the hash 
address. However, if the keys were larger and compart* 
sons took longer* the relative mix savings would closely 
approximate the savings in number of probes. 

Communication* December 1982 

oT Volume 23 

the ACM Number 12 



Fig. 3. The average number of probes per unsuccessful acarth, as M 
and M ' -* ob, for coalesced hashing <C„ Co* Cerr for 0 - 1. Q.86, 
0wt\ separate chaining (S)> separate chaining vilh ordered chains 
(SO), tinear probing (L), and double hashing (D). 




5.1 Standard Coalesced Hashing (d ) 

Standard coalesced hashing is the special case of 
coalesced hashing for which fi » 1 and there is no cellar. 
This is obviously the most realistic comparison that can 
be made, because except for the initialization of the 
address region size, standard coalesced hashing and 

Fig. 5. The average uix execution time per unsuccessful search, as 
M ' - °°. for coalesced hashing <C„ da., Q»t for 0 • 1, 0.86, /W), 
separate chaining (S), separate chaining with ordered chains (SOX 
linear probing (L), and donbie hashing (D). 




M AT U 0.9 U 



F<g.4. The average number ofprobes per successful scarcfa, ex A# and 

separate chaining ($X separate chaining with ordered chains (SO), 
hnear probing (L), and double hashing (D). 




"tuned" coalesced hashing are identical Figures 3 and 
4 show that the savings in number ofprobes per search 
can be as much as 14 percent (unsuccessful) and 6 
percent (successful). In Figs. 5 and 6, the corresponding 
savings in mix searching time is 6 percent (unsuccessful) 
and 2 percent (successful). 

Fig. 6. The average mix execution time per successful search, as 
for coalesced hashing <C„ Co*, Corr for fi • 1, 0.86, Parrl 
separate chaining (S). separate chaining with ordered chains (SOl 
linear probing (L). and double hashing (D). 
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&2 Separate (or Direct) Chaining (S) 

The separate chaining method is given an unfair 
advantage in Figs. 3 and 4: the number of probes per 
search is graphed as a function of a « N/M rather than 
or « N/M' and does not take into account the number of 
auxiliary slots used to store colliders. In order to make 
the comparison fair, we must adjust the load factor 
accordingly. 

Separate chaining implementations are designed of- 
ten to accommodate about N = M records; an average 
of M(\ - \IMY* s Mfe auxiliary slots are needed to 
store the colliders. The total table size is thus U ' s M 
+ Mfe. Solving backwards for M t we get M = 0.73 W. 
In other words, we may consider separate chaining to be 
the special case of coalesced hashing for which fi - 
0.731, except that no more records can be inserted once 
the cellar overflows. Hence, the adjusted load factor is 
a » 0.731a, and overflow occurs when there are around 
N-A/a 0.731 M' inserted records. (This is a reasonable 
space/time compromise: if we make M smaller, then 
more records can usually be stored before overflow 
occurs, but the average search times blow up; if we 
increase M to get better search times, then overflow 
occurs much sooner, and many slots are wasted.) 

If we adjust the load factors in Figs. 3 and 4 in this 
way, Algorithm C generates better search statistics: the 
expected number of probes per search for separate chain- 
ing is v 1.37 (unsuccessful) and = L5 (successful) when 
the load factor a is I, while that for coalesced hashing is 
a; 132 (unsuccessful) and a 1.44 (successful) when the 
load factor a — Pa is equal to 0.731. 

The graphs in Figs. 5 and 6 already reflect this load 
factor adjustment In fact, the mix implementation of 
separate chaining (Program S in [10]) is identical to 
Program C, except that & is initialized to 0.731 and 
overflow is signaled automatically when the cellar runs 
out of empty slots. Program C is slightly quicker in mix 
execution time than Program S* but more importantly, 
the coalesced hashing implementation is more space 
efficient: Program S usually overflows when a = 0.731, 
while Program C can always obtain full storage utiliza- 
tion a ■» 1. This confirms our intuition that coalesced 
hashing can accomodate more records than the separate 
chaining method and still outperform separate chaining 
before that method overflows. 

S3 Separate Chaining with Ordered Chains (SO) 

This method is a variation of separate chaining in 
which the chains are kept ordered by key value. The 
expected number of probes per successful search does 
not change, but unsuccessful searches are slightly 
quicker, because only about half the chain needs to be 
searched, on the average. 

Our remarks about adjusting the bad factor in Figs. 
3 and 4 also apply to method SO. But even after that is 
done, the average number of probes per unsuccessful 
search as well as the expected mix unsuccessful search 
rime is slightly better for this method than for coalesced 
hashing. However, as Fig. 6 illustrates, the average suc- 



cessful search time of Program SO is worse than Program 
Cs, and in real-life situations, the difference is likely to 
be more apparent, because records that are inserted first 
tend to be looked up more often and should be kept near 
the beginning of the chain, not rearranged. 

Method SO has the same storage limitations as the 
separate chaining scheme (le., the table usually over- 
flows when N « Af s 0.731 A/'), whereas coalesced 
hashing can obtain full storage utilization, 

SA linear Probing (L) and Doable Hashing (D) 

When searching for a record with key K, the linear 
probing method first checks location hash(K), and if 
another record is already there, it steps cyclically through 
the table, starting at location hash(K% until the record is 
found (successful search) or an empty slot is reached 
(unsuccessful search). Insertions are done by placing the 
record into the empty slot that terminated the unsuc- 
cessful search. Double hashing generalizes this by letting 
the cyclic step size be a function of JT. 
• We have to adjust the load factor in the opposite 
direction when we compare Algorithm C with methods 
L and D, because the latter do not require LINK fields. 
For example, if we suppose that the LINK field com- 
prises { of the total record size in a coalesced hashing 
implementation, then the search statistics in Figs. 3 and 
4 for Algorithm C with load factor a should be compared 
against those for linear probing and double hashing with 
load factor (J )cl In this case, the average number of 
probes per search is still better for coalesced hashing. 

However, the LINK Geld is often much smaller than 
the rest of the record, and sometimes it can be included 
in the table at virtually no extra cost The mix imple- 
mentation Program C in (10] assumes that the mix field 
can be squeezed into the record without need of extra 
storage space. Figures S and 6, therefore, require no load 
factor adjustment 

To balance matters, the mix implementations of lin- 
ear probing and double hashing, which are given in [10] 
and [7], contain two code optimizations. First, since 
LINK fields are not used in methods L and D, we no 
longer need 0 to denote a null LINK* and we can 
renumber the table slots from 0 to Af ' - 1; the hash 
function now returns a value between 0 and M f - 1. 
This makes the hash address computation faster by la, 
because the instruction INC I, 1 can be eliminated. 
Second, the empty slots are denoted by the value 0 in 
order to make the comparisons in the inner loop as fast 
as possible. This means that records are not allowed to 
have a key value of 0. The final results are graphed in 
Figs. 5 and 6. Coalesced hashing dearly dominates when 
the load factor is greater than 0.6. 



fi. Deletions 

It is often useful in hashing applications to be able to 
delete records when they no longer logically belong to 
the set of objects being represented in the hash table. For 
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example, in an airlines reservations system, passenger 
records are often expunged soon after the flight has 
taken place. 

One possible deletion strategy often used for linear 
probing and double hashing is to include a special one- 
bit DELETED field in each record that says whether or 
not the record has been deleted. The search algorithm 
most be modified to treat each "deleted" table slot as if 
it were occupied by a null record, even though the entire 
record is still there. This is especially desirable when 
there are pointers to the records from outside the table. 

If there are no such external pointers to worry about, 
the "deleted" table slots can be reused for latex insertions: 
Whenever an empty slot is needed in step C5 of Algo- 
rithm C, the record is inserted into the first "deleted" 
slot encountered during the unsuccessful search; if there 
is no such slot, an empty slot is allocated in the usual 
way. However, a certain percentage of the "deleted" slots 
probably will remain unused, thus preventing full storage 
utilization. Also, insertions and deletions over a pro- 
longed period would cause the expected search times to 
approximate those for a full table, regardless of the 
number of undeleted records, because the "deleted" 
records make the searches longer. 

If we are willing to spend a little extra time per 
deletion, Tve can do without the DELETED Held by 
relocating some of the records that follow in the chain. 
The basic idea is this: First, we find the record we want 
to delete, mark its table slot empty, and set the LINK 
field of its predecessor (if any) to the null value 0. Then 
we use Algorithm C to reinsert each record in the re- 
mainder of the chain, but whenever an empty slot is 
needed in step CS, we use the position that the record 
already occupies. 

This method can be illustrated by deleting al from 
location 10 in Fig. 7(a); the end result is pictured in Fig. 
7(b). The first step is to create a hole in position 10 where 
al was, and to set Audrey's LINK field to 0. Then we 
process the remainder of the chain. The next record 

Fig. 7. («) Inserting ihc tajhl records; (b) Inserting all the records except > 

(a) 



TOOTiE rehashes to the hole in location 10, so tootie 
moves up to plug the hole* leaving a new hole in position 
9. Next, donna collides with audrey during rehashing, 
so donna remains in slot 8 and is linked to Audrey. 
Then mark also collides with Audrey; we leave mark in 
position 7 and link it to donna, which was formerly at 
the end of Audrey's hash chain. The record jeff rehashes 
to the hole in slot 9, so we move it up to plug the hole, 
and a new hole appears in position 6. Finally, dave 
rehashes to position 9 and joins jefp's chain. 

Location 6 is the current hole position when the 
deletion algorithm terminates, so we set EMPTY[€[ <- 
true and return it to the pool of empty slots. However, 
the value of A in Algorithm C is already 5, so step C5 
wul never try to reuse location 6 when an empty slot is 
needed. 

We can solve this problem by using an available' 
space list in step CS rather than the variable R; the list 
must be doubly linked so that a slot can be removed 
quickly from the list in step C6\ The available-space list 
does not require any extra space per table slot, since we 
can use the KEY and LINK fields of the empty slots for 
the two pointer fields. (The KEY field is much larger 
than the LINK field in typical implementations.) For 
clarity, we rename the two pointer fields NEXT and 
PREV. Slot 0 in the table acts as the dummy start of the 
available-space list, so NEXT[0] points to the first actual 
slot in the list and PREV[Q\ points to the last. Before 
any records are inserted into the table, the following 
extra initializations must be made: NEXT[0] <- M' 
PREV\M'\ «- 0; and NEXT[i) <- # - I and PREV[i - 
1] 4-4 for 1 <i< A/'. We replace steps C5 and C6 by 

CS. [Find empty slot] (The search for if in the chain 
was unsuccessful, so we will try to find an empty 
table slot to store K) If the table is already full (Le., 
NEXT[Q] = 0), the algorithm terminates with over- 
flow. Otherwise, set LINK[i] «- NEXT[0] and l+- 
NEXT[Q]. 

C& [Insert new record,] Remove the ith slot from the 
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